diff options
author | Jisi Liu <jisi.liu@gmail.com> | 2017-07-18 16:26:16 -0700 |
---|---|---|
committer | Jisi Liu <jisi.liu@gmail.com> | 2017-07-18 16:26:16 -0700 |
commit | 3af881c7e2db0553207876acbdb91c6e77974e44 (patch) | |
tree | 35e2a96cce802d4235bd29afa396e28ead892fa2 /python | |
parent | 86975301f1876ce1934612777b8ca6c76520f5cc (diff) | |
parent | 942a29cecd36f2a4b22fdd2179635cd548e6bd27 (diff) |
Merge master into 3.4.x
Diffstat (limited to 'python')
-rw-r--r-- | python/README.md | 83 | ||||
-rw-r--r-- | python/release/wheel/Dockerfile | 6 | ||||
-rw-r--r-- | python/release/wheel/README.md | 17 | ||||
-rwxr-xr-x | python/release/wheel/build_wheel_manylinux.sh | 27 | ||||
-rwxr-xr-x | python/release/wheel/protobuf_optimized_pip.sh | 66 | ||||
-rw-r--r-- | python/tox.ini | 1 |
6 files changed, 162 insertions, 38 deletions
diff --git a/python/README.md b/python/README.md index 8f3db785..4c194297 100644 --- a/python/README.md +++ b/python/README.md @@ -32,77 +32,84 @@ Installation 1) Make sure you have Python 2.6 or newer. If in doubt, run: - $ python -V + $ python -V 2) If you do not have setuptools installed, note that it will be - downloaded and installed automatically as soon as you run setup.py. + downloaded and installed automatically as soon as you run `setup.py`. If you would rather install it manually, you may do so by following - the instructions on this page: + the instructions on [this page](https://packaging.python.org/en/latest/installing.html#setup-for-installing-packages). - https://packaging.python.org/en/latest/installing.html#setup-for-installing-packages - -3) Build the C++ code, or install a binary distribution of protoc. If +3) Build the C++ code, or install a binary distribution of `protoc`. If you install a binary distribution, make sure that it is the same version as this package. If in doubt, run: - $ protoc --version + $ protoc --version 4) Build and run the tests: - $ python setup.py build - $ python setup.py test + $ python setup.py build + $ python setup.py test + + To build, test, and use the C++ implementation, you must first compile + `libprotobuf.so`: + + $ (cd .. && make) + + On OS X: + + If you are running a Homebrew-provided Python, you must make sure another + version of protobuf is not already installed, as Homebrew's Python will + search `/usr/local/lib` for `libprotobuf.so` before it searches + `../src/.libs`. - To build, test, and use the C++ implementation, you must first compile - libprotobuf.so: + You can either unlink Homebrew's protobuf or install the `libprotobuf` you + built earlier: - $ (cd .. && make) + $ brew unlink protobuf - On OS X: + or - If you are running a homebrew-provided python, you must make sure another - version of protobuf is not already installed, as homebrew's python will - search /usr/local/lib for libprotobuf.so before it searches ../src/.libs - You can either unlink homebrew's protobuf or install the libprotobuf you - built earlier: + $ (cd .. && make install) - $ brew unlink protobuf - or - $ (cd .. && make install) + On other *nix: - On other *nix: + You must make `libprotobuf.so` dynamically available. You can either + install libprotobuf you built earlier, or set `LD_LIBRARY_PATH`: - You must make libprotobuf.so dynamically available. You can either - install libprotobuf you built earlier, or set LD_LIBRARY_PATH: + $ export LD_LIBRARY_PATH=../src/.libs - $ export LD_LIBRARY_PATH=../src/.libs - or - $ (cd .. && make install) + or - To build the C++ implementation run: - $ python setup.py build --cpp_implementation + $ (cd .. && make install) - Then run the tests like so: - $ python setup.py test --cpp_implementation + To build the C++ implementation run: + + $ python setup.py build --cpp_implementation + + Then run the tests like so: + + $ python setup.py test --cpp_implementation If some tests fail, this library may not work correctly on your system. Continue at your own risk. Please note that there is a known problem with some versions of Python on Cygwin which causes the tests to fail after printing the - error: "sem_init: Resource temporarily unavailable". This appears - to be a bug either in Cygwin or in Python: - http://www.cygwin.com/ml/cygwin/2005-07/msg01378.html + error: `sem_init: Resource temporarily unavailable`. This appears + to be a [bug either in Cygwin or in + Python](http://www.cygwin.com/ml/cygwin/2005-07/msg01378.html). + We do not know if or when it might be fixed. We also do not know how likely it is that this bug will affect users in practice. 5) Install: - $ python setup.py install + $ python setup.py install - or: + or: - $ (cd .. && make install) - $ python setup.py install --cpp_implementation + $ (cd .. && make install) + $ python setup.py install --cpp_implementation This step may require superuser privileges. NOTE: To use C++ implementation, you need to export an environment diff --git a/python/release/wheel/Dockerfile b/python/release/wheel/Dockerfile new file mode 100644 index 00000000..f38ec2f5 --- /dev/null +++ b/python/release/wheel/Dockerfile @@ -0,0 +1,6 @@ +FROM quay.io/pypa/manylinux1_x86_64 + +RUN yum install -y libtool +RUN /opt/python/cp27-cp27mu/bin/pip install twine + +COPY protobuf_optimized_pip.sh / diff --git a/python/release/wheel/README.md b/python/release/wheel/README.md new file mode 100644 index 00000000..edda2cd7 --- /dev/null +++ b/python/release/wheel/README.md @@ -0,0 +1,17 @@ +Description +------------------------------ +This directory is used to build released wheels according to PEP513 and upload +them to pypi. + +Usage +------------------------------ +For example, to release 3.3.0: + ./protobuf_optimized_pip.sh 3.3.0 PYPI_USERNAME PYPI_PASSWORD + +Structure +------------------------------ +| Source | Source | +|--------------------------------------|---------------------------------------------------| +| protobuf_optimized_pip.sh | Entry point. Calling Dockerfile and build_wheel_manylinux.sh | +| Dockerfile | Build docker image according to PEP513. | +| build_wheel_manylinux.sh | Build wheel packages in the docker container. | diff --git a/python/release/wheel/build_wheel_manylinux.sh b/python/release/wheel/build_wheel_manylinux.sh new file mode 100755 index 00000000..39fd8c12 --- /dev/null +++ b/python/release/wheel/build_wheel_manylinux.sh @@ -0,0 +1,27 @@ +#!/bin/bash + +# Print usage and fail. +function usage() { + echo "Usage: protobuf_optimized_pip.sh PROTOBUF_VERSION PYPI_USERNAME PYPI_PASSWORD" >&2 + exit 1 # Causes caller to exit because we use -e. +} + +# Validate arguments. +if [ $0 != ./build_wheel_manylinux.sh ]; then + echo "Please run this script from the directory in which it is located." >&2 + exit 1 +fi + +if [ $# -lt 3 ]; then + usage + exit 1 +fi + +PROTOBUF_VERSION=$1 +PYPI_USERNAME=$2 +PYPI_PASSWORD=$3 + +docker rmi protobuf-python-wheel +docker build . -t protobuf-python-wheel +docker run --rm protobuf-python-wheel ./protobuf_optimized_pip.sh $PROTOBUF_VERSION $PYPI_USERNAME $PYPI_PASSWORD +docker rmi protobuf-python-wheel diff --git a/python/release/wheel/protobuf_optimized_pip.sh b/python/release/wheel/protobuf_optimized_pip.sh new file mode 100755 index 00000000..469661aa --- /dev/null +++ b/python/release/wheel/protobuf_optimized_pip.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash + +# DO NOT use this script manually! Called by docker. + +set -ex + +# Print usage and fail. +function usage() { + echo "Usage: protobuf_optimized_pip.sh PROTOBUF_VERSION PYPI_USERNAME PYPI_PASSWORD" >&2 + exit 1 # Causes caller to exit because we use -e. +} + +# Build wheel +function build_wheel() { + PYTHON_VERSION=$1 + PYTHON_BIN=/opt/python/${PYTHON_VERSION}/bin/python + + $PYTHON_BIN setup.py bdist_wheel --cpp_implementation --compile_static_extension + auditwheel repair dist/protobuf-${PROTOBUF_VERSION}-${PYTHON_VERSION}-linux_x86_64.whl +} + +# Validate arguments. +if [ $0 != ./protobuf_optimized_pip.sh ]; then + echo "Please run this script from the directory in which it is located." >&2 + exit 1 +fi + +if [ $# -lt 3 ]; then + usage + exit 1 +fi + +PROTOBUF_VERSION=$1 +PYPI_USERNAME=$2 +PYPI_PASSWORD=$3 + +DIR=${PWD}/'protobuf-python-build' +PYTHON_VERSIONS=('cp27-cp27mu' 'cp33-cp33m' 'cp34-cp34m' 'cp35-cp35m' 'cp36-cp36m') + +mkdir -p ${DIR} +cd ${DIR} +curl -SsL -O https://github.com/google/protobuf/archive/v${PROTOBUF_VERSION}.tar.gz +tar xzf v${PROTOBUF_VERSION}.tar.gz +cd $DIR/protobuf-${PROTOBUF_VERSION} + +# Autoconf on centos 5.11 cannot recognize AC_PROG_OBJC. +sed -i '/AC_PROG_OBJC/d' configure.ac +sed -i 's/conformance\/Makefile//g' configure.ac + +# Build protoc +./autogen.sh +CXXFLAGS="-fPIC -g -O2" ./configure +make -j8 +export PROTOC=$DIR/src/protoc + +cd python + +for PYTHON_VERSION in "${PYTHON_VERSIONS[@]}" +do + build_wheel $PYTHON_VERSION +done + +/opt/python/cp27-cp27mu/bin/twine upload wheelhouse/* <<! +$PYPI_USERNAME +$PYPI_PASSWORD +! diff --git a/python/tox.ini b/python/tox.ini index 1600db21..baa96dba 100644 --- a/python/tox.ini +++ b/python/tox.ini @@ -9,6 +9,7 @@ setenv = cpp: LD_LIBRARY_PATH={toxinidir}/../src/.libs cpp: DYLD_LIBRARY_PATH={toxinidir}/../src/.libs cpp: PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp + python: PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python commands = python setup.py -q build_py python: python setup.py -q build |