Deploy maven artifacts to Bintray as well.

PiperOrigin-RevId: 167589397

4 files changed, 205 insertions, 76 deletions

diff --git a/tensorflow/java/maven/README.md b/tensorflow/java/maven/README.md
index 17bb799961..6227775361 100644
--- a/tensorflow/java/maven/README.md
+++ b/tensorflow/java/maven/README.md
@@ -1,11 +1,13 @@
 # TensorFlow for Java using Maven
 
-The [TensorFlow Java
-API](https://www.tensorflow.org/api_docs/java/reference/org/tensorflow/package-summary)
-is available through artifacts uploaded to [Maven
-Central](https://oss.sonatype.org/content/repositories/snapshots/org/tensorflow/).
-This document describes the process of updating the release artifacts. It does
-_not_ describe how to use the artifacts, for which the reader is referred to the
+The
+[TensorFlow Java API](https://www.tensorflow.org/api_docs/java/reference/org/tensorflow/package-summary)
+is available on Maven Central and JCenter through artifacts uploaded to
+[OSS Sonatype](https://oss.sonatype.org/content/repositories/releases/org/tensorflow/) and
+[Bintray](https://bintray.com/google/tensorflow/tensorflow) respectively. This
+document describes the process of updating the release artifacts. It does _not_
+describe how to use the artifacts, for which the reader is referred to
+the
 [TensorFlow for Java installation instructions](https://www.tensorflow.org/code/tensorflow/java/README.md).
 
 ## Background
@@ -20,7 +22,7 @@ Hence, the process for building and uploading release artifacts is not a single
 
 ## Artifact Structure
 
-There are five artifacts and thus `pom.xml`s involved in this release:
+There are six artifacts and thus `pom.xml`s involved in this release:
 
 1.  `tensorflow`: The single dependency for projects requiring TensorFlow for
     Java. This convenience package depends on the two below, and is the one that
@@ -37,16 +39,22 @@ There are five artifacts and thus `pom.xml`s involved in this release:
 4.  `proto`: Generated Java code for TensorFlow protocol buffers
     (e.g., `MetaGraphDef`, `ConfigProto` etc.)
 
-5.  [`parentpom`](https://maven.apache.org/pom/index.html): Common settings
+5. `tensorflow-android`: A package geared towards
+    supporting [TensorFlow on Android](../../contrib/android/README.md), and is
+    a self-contained Android AAR library containing all necessary native and
+    Java code.
+
+6.  [`parentpom`](https://maven.apache.org/pom/index.html): Common settings
     shared by all of the above.
 
+
 ## Updating the release
 
-The TensorFlow artifacts at Maven Central are created from files built as part
-of the TensorFlow release process (which uses `bazel`). The author's lack of
-familiarity with Maven best practices combined with the use of a different build
-system means that this process is possibly not ideal, but it's what we've got.
-Suggestions are welcome.
+The Maven artifacts are created from files built as part of the TensorFlow
+release process (which uses `bazel`). The author's lack of familiarity with
+Maven best practices combined with the use of a different build system means
+that this process is possibly not ideal, but it's what we've got.  Suggestions
+are welcome.
 
 In order to isolate the environment used for building, all release processes are
 conducted in a [Docker](https://www.docker.com) container.
@@ -59,16 +67,28 @@ conducted in a [Docker](https://www.docker.com) container.
     account does not have permissions, then you'll need to ask someone who does
     to [file a ticket](https://issues.sonatype.org/) to add to the permissions
     ([sample ticket](https://issues.sonatype.org/browse/MVNCENTRAL-1637)).
--   A GPG signing key, required [to sign the release artifacts](http://central.sonatype.org/pages/apache-maven.html#gpg-signed-components).
-
-### Deploying to Maven Central
-
-1.  Create a file with your OSSRH credentials (or perhaps you use `mvn` and have
-    it in `~/.m2/settings.xml`):
+-   An account at [bintray.com](https://bintray.com) that has permissions to
+    update the [tensorflow repository](https://bintray.com/google/tensorflow).
+    If your account does not have permissions, then you'll need to ask one of
+    the [organization administrators](https://bintray.com/google) to give you
+    permissions to update the `tensorflow` repository. Please keep the
+    [repository option](https://bintray.com/google/tensorflow/edit?tab=general)
+    to *"GPG sign uploaded files using Bintray's public/private key pair"*
+    **unchecked**, otherwise it will conflict with locally signed artifacts.
+-   A GPG signing key, required
+    [to sign the release artifacts](http://central.sonatype.org/pages/apache-maven.html#gpg-signed-components).
+
+### Deploying to Sonatype and Bintray
+
+1.  Create a file with your OSSRH credentials and
+    [Bintray API key](https://bintray.com/docs/usermanual/interacting/interacting_interacting.html#anchorAPIKEY)
+    (or perhaps you use `mvn` and have it in `~/.m2/settings.xml`):
 
     ```sh
     SONATYPE_USERNAME="your_sonatype.org_username_here"
     SONATYPE_PASSWORD="your_sonatype.org_password_here"
+    BINTRAY_USERNAME="your_bintray_username_here"
+    BINTRAY_API_KEY="your_bintray_api_key_here"
     GPG_PASSPHRASE="your_gpg_passphrase_here"
     cat >/tmp/settings.xml <<EOF
     <settings>
@@ -78,19 +98,16 @@ conducted in a [Docker](https://www.docker.com) container.
           <username>${SONATYPE_USERNAME}</username>
           <password>${SONATYPE_PASSWORD}</password>
         </server>
+        <server>
+          <id>bintray</id>
+          <username>${BINTRAY_USERNAME}</username>
+          <password>${BINTRAY_API_KEY}</password>
+        </server>
       </servers>
-      <profiles>
-        <profile>
-          <id>ossrh</id>
-          <activation>
-            <activeByDefault>true</activeByDefault>
-          </activation>
-          <properties>
-            <gpg.executable>gpg2</gpg.executable>
-            <gpg.passphrase>${GPG_PASSPHRASE}</gpg.passphrase>
-          </properties>
-        </profile>
-      </profiles>
+      <properties>
+        <gpg.executable>gpg2</gpg.executable>
+        <gpg.passphrase>${GPG_PASSPHRASE}</gpg.passphrase>
+      </properties>
     </settings>
     EOF
     ```
@@ -98,30 +115,83 @@ conducted in a [Docker](https://www.docker.com) container.
 2.  Run the `release.sh` script.
 
 3.  If the script above succeeds then the artifacts would have been uploaded to
-    the private staging repository. After verifying the release, visit
-    https://oss.sonatype.org/#stagingRepositories, find the `org.tensorflow`
+    the private staging repository in Sonatype, and as unpublished artifacts in
+    Bintray. After verifying the release, you should finalize or abort the
+    release on both sites.
+
+4.  Visit https://oss.sonatype.org/#stagingRepositories, find the `org.tensorflow`
     release and click on either `Release` to finalize the release, or `Drop` to
-    abort. Some things of note:
+    abort.
+
+5.  Visit https://bintray.com/google/tensorflow/tensorflow, and select the
+    version you just uploaded. Notice there's a message about unpublished
+    artifacts. Click on either `Publish` to finalize the release, or `Discard`
+    to abort.
 
+6.  Some things of note:
     - For details, look at the [Sonatype guide](http://central.sonatype.org/pages/releasing-the-deployment.html).
     - Syncing with [Maven Central](http://repo1.maven.org/maven2/org/tensorflow/)
       can take 10 minutes to 2 hours (as per the [OSSRH
       guide](http://central.sonatype.org/pages/ossrh-guide.html#releasing-to-central)).
+    - For Bintray details, refer to their guide on
+      [managing uploaded content](https://bintray.com/docs/usermanual/uploads/uploads_managinguploadedcontent.html#_publishing).
 
-4.  Upon successful release, commit changes to all the `pom.xml` files
+7.  Upon successful release, commit changes to all the `pom.xml` files
     (which should have the updated version number).
 
 ### Snapshots
 
 If the `TF_VERSION` provided to the `release.sh` script ends in `-SNAPSHOT`,
 then instead of using official release files, the nightly build artifacts from
-https://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/ and
-https://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow-windows/ will
-be used to upload to the Maven Central snapshots repository.
+https://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/,
+https://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow-windows/ and
+https://ci.tensorflow.org/view/Nightly/job/nightly-android
+will be used to upload to the Maven Central snapshots repository. (Note that
+snapshots are only uploaded to Maven Central, not Bintray.)
+
+### Skip deploying to a repository
+
+Should you need, setting environment variables `DEPLOY_OSSRH=0` or
+`DEPLOY_BINTRAY=0` when calling `release.sh` will skip deploying to OSSRH or
+Bintray respectively. Note that snapshots are only uploaded to OSSRH, so you
+cannot skip deploying to OSSRH for a `-SNAPSHOT` version.
+
+## The overall flow
+
+This section provides some pointers around how artifacts are currently
+assembled.
+
+All native and java code is first built and tested on
+a [Tensorflow Jenkins server](https://ci.tensorflow.org/) which run various
+scripts under the [`tools/ci_build`](../../tools/ci_build/) directory. Of
+particular interest may be `tools/ci_build/builds/libtensorflow.sh` which
+bundles Java-related build sources and outputs into archives, and
+`tools/ci_build/builds/android_full.sh` which produces an Android AAR package.
+
+Maven artifacts however are not created in Jenkins. Instead, artifacts are
+created and deployed externally on-demand, when a maintainer runs the
+`release.sh` script.
+
+This script spins up a Docker instance which downloads the archives created by
+successful runs of various `tools/ci_build` scripts on the Tensorflow Jenkins
+server.
+
+It organizes these archives locally into a maven-friendly layout, and runs `mvn
+deploy` to create maven artifacts within the container. Native libraries built
+in Jenkins are used as-is, but srcjars for java code are used to compile class
+files and generate javadocs.) It also downloads the Android AAR from the Jenkins
+server and directly deploys it via `mvn gpg:sign-and-deploy-file`.
+
+`release.sh` then stages these artifacts to OSSRH and Bintray, and if all goes
+well a maintainer can log into both sites to promote them as a new release.
 
+There is a small change to the flow for a standard (rather than a `-SNAPSHOT`)
+release. Rather than downloading archives directly from jobs on the Jenkins
+server, the script uses a static repository of QA-blessed archives.
 
 ## References
 
 -   [Sonatype guide](http://central.sonatype.org/pages/ossrh-guide.html) for
     hosting releases.
 -   [Ticket that created the `org/tensorflow` configuration](https://issues.sonatype.org/browse/OSSRH-28072) on OSSRH.
+-   The [Bintray User Manual](https://bintray.com/docs/usermanual/index.html)
diff --git a/tensorflow/java/maven/pom.xml b/tensorflow/java/maven/pom.xml
index cc4fbc4a75..0a3552d756 100644
--- a/tensorflow/java/maven/pom.xml
+++ b/tensorflow/java/maven/pom.xml
@@ -33,18 +33,35 @@
     <module>proto</module>
   </modules>
 
-
-  <!-- Requirements from http://central.sonatype.org/pages/apache-maven.html -->
-  <distributionManagement>
-    <snapshotRepository>
-      <id>ossrh</id>
-      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
-    </snapshotRepository>
-    <repository>
+  <!-- Two profiles are used:
+       ossrh - deploys to ossrh/maven central
+       bintray - deploys to bintray/jcenter. -->
+  <profiles>
+    <profile>
       <id>ossrh</id>
-      <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
-    </repository>
-  </distributionManagement>
+      <distributionManagement>
+        <!-- Sonatype requirements from http://central.sonatype.org/pages/apache-maven.html -->
+        <snapshotRepository>
+          <id>ossrh</id>
+          <url>https://oss.sonatype.org/content/repositories/snapshots</url>
+        </snapshotRepository>
+        <repository>
+          <id>ossrh</id>
+          <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
+        </repository>
+      </distributionManagement>
+    </profile>
+    <profile>
+      <id>bintray</id>
+      <distributionManagement>
+        <!-- https://blog.bintray.com/2015/09/17/publishing-your-maven-project-to-bintray/ -->
+        <repository>
+          <id>bintray</id>
+          <url>https://api.bintray.com/maven/google/tensorflow/tensorflow/;publish=0</url>
+        </repository>
+      </distributionManagement>
+    </profile>
+  </profiles>
   <!-- http://central.sonatype.org/pages/requirements.html#developer-information -->
   <developers>
     <developer>
@@ -55,19 +72,6 @@
   </developers>
   <build>
     <plugins>
-      <!-- Staging repository configuration: http://central.sonatype.org/pages/apache-maven.html#nexus-staging-maven-plugin-for-deployment-and-release -->
-      <plugin>
-        <groupId>org.sonatype.plugins</groupId>
-        <artifactId>nexus-staging-maven-plugin</artifactId>
-        <version>1.6.7</version>
-        <extensions>true</extensions>
-        <configuration>
-           <serverId>ossrh</serverId>
-           <nexusUrl>https://oss.sonatype.org/</nexusUrl>
-
-           <autoReleaseAfterClose>false</autoReleaseAfterClose>
-        </configuration>
-      </plugin>
       <!-- GPG signed components: http://central.sonatype.org/pages/apache-maven.html#gpg-signed-components -->
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
diff --git a/tensorflow/java/maven/release.sh b/tensorflow/java/maven/release.sh
index b95a4d4674..9012ea14ea 100755
--- a/tensorflow/java/maven/release.sh
+++ b/tensorflow/java/maven/release.sh
@@ -49,6 +49,8 @@ fi
 set -ex
 docker run \
   -e TF_VERSION="${TF_VERSION}" \
+  -e DEPLOY_OSSRH="${DEPLOY_OSSRH:-true}" \
+  -e DEPLOY_BINTRAY="${DEPLOY_BINTRAY:-true}" \
   -v ${PWD}:/tensorflow \
   -v "${SETTINGS_XML}":/root/.m2/settings.xml \
   -v ${HOME}/.gnupg:/root/.gnupg \
diff --git a/tensorflow/java/maven/run_inside_container.sh b/tensorflow/java/maven/run_inside_container.sh
index 6b4d5d7032..a2ce097195 100644
--- a/tensorflow/java/maven/run_inside_container.sh
+++ b/tensorflow/java/maven/run_inside_container.sh
@@ -19,11 +19,23 @@
 
 
 RELEASE_URL_PREFIX="https://storage.googleapis.com/tensorflow/libtensorflow"
+
+# By default we deploy to both ossrh and bintray. These two
+# environment variables can be set to skip either repository.
+DEPLOY_BINTRAY="${DEPLOY_BINTRAY:-true}"
+DEPLOY_OSSRH="${DEPLOY_OSSRH:-true}"
+
 IS_SNAPSHOT="false"
 if [[ "${TF_VERSION}" == *"-SNAPSHOT" ]]; then
   IS_SNAPSHOT="true"
+  # Bintray does not allow snapshots.
+  DEPLOY_BINTRAY="false"
 fi
 PROTOC_RELEASE_URL="https://github.com/google/protobuf/releases/download/v3.3.0/protoc-3.3.0-linux-x86_64.zip"
+if [[ "${DEPLOY_BINTRAY}" != "true" && "${DEPLOY_OSSRH}" != "true" ]]; then
+  echo "Must deploy to at least one of Bintray or OSSRH" >&2
+  exit 2
+fi
 
 set -ex
 
@@ -39,6 +51,20 @@ update_version_in_pom() {
   mvn versions:set -DnewVersion="${TF_VERSION}"
 }
 
+# Fetch a property from pom files for a given profile.
+# Arguments:
+#   profile - name of the selected profile.
+#   property - name of the property to be retrieved.
+# Output:
+#   Echo property value to stdout
+mvn_property() {
+  local profile="$1"
+  local prop="$2"
+  mvn -q --non-recursive exec:exec -P "${profile}" \
+    -Dexec.executable='echo' \
+    -Dexec.args="\${${prop}}"
+}
+
 download_libtensorflow() {
   if [[ "${IS_SNAPSHOT}" == "true" ]]; then
     URL="http://ci.tensorflow.org/view/Nightly/job/nightly-libtensorflow/TYPE=cpu-slave/lastSuccessfulBuild/artifact/lib_package/libtensorflow-src.jar"
@@ -137,29 +163,50 @@ generate_java_protos() {
   rm -rf "${DIR}/proto/tmp"
 }
 
+# Deploy artifacts using a specific profile.
+# Arguments:
+#   profile - name of selected profile.
+# Outputs:
+#   n/a
+deploy_profile() {
+  local profile="$1"
+  # Deploy the non-android pieces.
+  mvn deploy -P"${profile}"
+  # Determine the correct pom file property to use
+  # for the repository url.
+  local rtype
+  if [[ "${IS_SNAPSHOT}" == "true" ]]; then
+    rtype='snapshotRepository'
+  else
+    rtype='repository'
+  fi
+  local url=$(mvn_property "${profile}" "project.distributionManagement.${rtype}.url")
+  local repositoryId=$(mvn_property "${profile}" "project.distributionManagement.${rtype}.id")
+  mvn gpg:sign-and-deploy-file \
+    -Dfile="${DIR}/tensorflow-android/target/tensorflow.aar" \
+    -DpomFile="${DIR}/tensorflow-android/target/pom-android.xml" \
+    -Durl="${url}" \
+    -DrepositoryId="${repositoryId}"
+}
+
 # If successfully built, try to deploy.
 # If successfully deployed, clean.
 # If deployment fails, debug with
 #   ./release.sh ${TF_VERSION} ${SETTINGS_XML} bash
 # To get a shell to poke around the maven artifacts with.
 deploy_artifacts() {
-  # This deploys the non-android pieces
-  mvn deploy
-
-  # Sign and deploy the previously downloaded aar file as a single
-  # maven artifact.
-  if [[ "${IS_SNAPSHOT}" == "true" ]]; then
-    REPO="https://oss.sonatype.org/content/repositories/snapshots"
-  else
-    REPO="https://oss.sonatype.org/service/local/staging/deploy/maven2/"
+  # Deploy artifacts to ossrh if requested.
+  if [[ "${DEPLOY_OSSRH}" == "true" ]]; then
+    deploy_profile 'ossrh'
+  fi
+  # Deploy artifacts to bintray if requested.
+  if [[ "${DEPLOY_BINTRAY}" == "true" ]]; then
+    deploy_profile 'bintray'
   fi
-  mvn gpg:sign-and-deploy-file -Dfile="${DIR}/tensorflow-android/target/tensorflow.aar" -DpomFile="${DIR}/tensorflow-android/target/pom-android.xml" -Durl=${REPO} -DrepositoryId=ossrh
-
   # Clean up when everything works
   clean
 }
 
-
 if [ -z "${TF_VERSION}" ]
 then
   echo "Must set the TF_VERSION environment variable"
@@ -189,8 +236,14 @@ set +ex
 if [[ "${IS_SNAPSHOT}" == "false" ]]; then
   echo "Uploaded to the staging repository"
   echo "After validating the release: "
-  echo "1. Login to https://oss.sonatype.org/#stagingRepositories"
-  echo "2. Find the 'org.tensorflow' staging release and click either 'Release' to release or 'Drop' to abort"
+  if [[ "${DEPLOY_OSSRH}" == "true" ]]; then
+    echo "* Login to https://oss.sonatype.org/#stagingRepositories"
+    echo "* Find the 'org.tensorflow' staging release and click either 'Release' to release or 'Drop' to abort"
+  fi
+  if [[ "${DEPLOY_BINTRAY}" == "true" ]]; then
+    echo "* Login to https://bintray.com/google/tensorflow/tensorflow"
+    echo "* Either 'Publish' unpublished items to release, or 'Discard' to abort"
+  fi
 else
   echo "Uploaded to the snapshot repository"
 fi