Merge pull request #9358 from jtattermusch/mingw_docs

Update docs on windows build

5 files changed, 50 insertions, 262 deletions

diff --git a/INSTALL.md b/INSTALL.md
index 686145566f..24f088ea49 100644
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -61,49 +61,45 @@ gRPC C Core library.
 There are several ways to build under Windows, of varying complexity depending
 on experience with the tools involved.
 
-<!--
-###Visual Studio
+###Pre-generated Visual Studio solution
 
-Versions 2013 and 2015 are both supported. You can use [their respective
-community
-editions](https://www.visualstudio.com/en-us/downloads/download-visual-studio-vs.aspx).
-
-Building the C Core:
-- Open [grpc.sln](https://github.com/grpc/grpc/blob/master/vsprojects/grpc.sln).
-- Select your build target.
-- Build the `grpc` project.
+The pre-generated VS projects & solution are checked into the repository under the [vsprojects](/vsprojects) directory.
+  
+###Building using CMake (with BoringSSL)
+- Install [CMake](https://cmake.org/download/).
+- Install [Active State Perl](http://www.activestate.com/activeperl/) (`choco install activeperl`)
+- Install [Ninja](https://ninja-build.org/) (`choco install ninja`)
+- Install [Go](https://golang.org/dl/) (`choco install golang`)
+- Install [yasm](http://yasm.tortall.net/) and add it to `PATH` (`choco install yasm`)
+- Run these commands in the repo root directory
+```
+> md .build
+> cd .build
+> call "%VS140COMNTOOLS%..\..\VC\vcvarsall.bat" x64
+> cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release
+> cmake --build .
+```
+NOTE: Currently you can only use Ninja to build using cmake on Windows (because of the boringssl dependency).
 
-Building the C++ runtime:
-- You need [CMake](https://cmake.org/) on your path to build protobuf (see below
-  for building using solely CMake).
-- Run `vsprojects/build_protos.bat` (needs `cmake.exe` in your path).
-- Open [buildtests_cxx.sln]()
-- Select your build target.
-- build the `grpc++` project.
--->
+###msys2 (with mingw)
 
-###msys2
+The Makefile (and source code) should support msys2's mingw32 and mingw64
+compilers. Building with msys2's native compiler is also possible, but
+difficult.
 
 This approach requires having [msys2](https://msys2.github.io/) installed.
+  
+```
+# Install prerequisites
+MSYS2$ pacman -S autoconf automake gcc libtool mingw-w64-x86_64-toolchain perl pkg-config zlib
+MSYS2$ pacman -S mingw-w64-x86_64-gflags
+```
 
-- The Makefile (and source code) should support msys2's mingw32 and mingw64
-  compilers. Building with msys2's native compiler is also possible, but
-  difficult.
-- The Makefile is expecting the Windows versions of OpenSSL (see
-  https://slproweb.com/products/Win32OpenSSL.html). It's also possible to build
-  the Windows version of OpenSSL from scratch. The output should be `libeay32`
-  and `ssleay32`.
-- If you are not installing the above files under msys2's path, you may specify
-  it, for instance, in the following way:
-  ```CPPFLAGS=”-I/c/OpenSSL-Win32/include” LDFLAGS=”-L/c/OpenSSL-Win32/lib” make static_c```
-- [protobuf3](https://github.com/google/protobuf/blob/master/src/README.md#c-installation---windows)
-  must be installed on the msys2 path.
-
-###Cmake (experimental)
+```
+# From mingw shell
+MINGW64$ export CPPFLAGS="-D_WIN32_WINNT=0x0600"
+MINGW64$ make
+```
 
-- Install [CMake](https://cmake.org/download/).
-- Run it over [grpc's
-  CMakeLists.txt](https://github.com/grpc/grpc/blob/master/CMakeLists.txt) to
-  generate "projects" for your compiler.
-- Build with your compiler of choice. The generated build files should have the
-  protobuf3 dependency baked in.
+NOTE: While most of the make targets are buildable under Mingw, some haven't been ported to Windows yet
+and may fail to build (mostly trying to include POSIX headers not available on Mingw).
diff --git a/templates/vsprojects/generate_debug_projects.sh b/templates/vsprojects/generate_debug_projects.sh
deleted file mode 100755
index f103ebe01a..0000000000
--- a/templates/vsprojects/generate_debug_projects.sh
+++ /dev/null
@@ -1,76 +0,0 @@
-#!/bin/sh
-
-# Copyright 2015, Google Inc.
-# All rights reserved.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are
-# met:
-#
-#     * Redistributions of source code must retain the above copyright
-# notice, this list of conditions and the following disclaimer.
-#     * Redistributions in binary form must reproduce the above
-# copyright notice, this list of conditions and the following disclaimer
-# in the documentation and/or other materials provided with the
-# distribution.
-#     * Neither the name of Google Inc. nor the names of its
-# contributors may be used to endorse or promote products derived from
-# this software without specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-# To properly use this, you'll need to add:
-#
-#    "debug": true
-#
-# to build.json
-
-cd `dirname $0`/../..
-
-git add .               #because we're using "git diff" to find changes to grpc.sln and then make files based on those changes, this prevents this file or other files from possibly being found in the diff
-
-./tools/buildgen/generate_projects-old.sh
-
-line_number=0
-
-git diff |
-grep -A2 \\+Project |       #find "Project" immediately after a backslash (escaped), plus 2 additional lines to capture the "libs = ", plus 1 line of "--".  matches will come from the generated grpc.sln
-while read p ; do
-  line_number=$((line_number + 1))
-  if [ "$line_number" -gt "4" ]; then
-    line_number=1;
-  fi
-  echo $line_number
-  echo $p
-  if [ "$line_number" -eq "1" ]; then
-    project_name=$(echo "$p" | cut -d\" -f 4)          #sed: extract line N only; cut with delimiter: ".  select only field 4
-  fi
-  if [ "$line_number" -eq "3" ]; then
-    lib_setting=$(echo "$p" | cut -d\" -f 2)          #
-    echo "project_name"
-    echo $project_name
-    echo "lib_setting"
-    echo $lib_setting
-    mkdir -p templates/vsprojects/$project_name
-    if [ "$lib_setting" = "True" ]; then
-      echo "lib: true"
-      echo '<%namespace file="../vcxproj_defs.include" import="gen_project"/>${gen_project("'$project_name'", libs)}' > templates/vsprojects/$project_name/$project_name.vcxproj.template
-    else
-      echo "lib: not true"
-      echo '<%namespace file="../vcxproj_defs.include" import="gen_project"/>${gen_project("'$project_name'", targets)}' > templates/vsprojects/$project_name/$project_name.vcxproj.template
-    fi
-  fi
- # sleep .5     #for testing
-done
-
-./tools/buildgen/generate_projects-old.sh
diff --git a/tools/buildgen/generate_projects-old.sh b/tools/buildgen/generate_projects-old.sh
deleted file mode 100644
index 55d93d4942..0000000000
--- a/tools/buildgen/generate_projects-old.sh
+++ /dev/null
@@ -1,76 +0,0 @@
-#!/bin/sh
-# Copyright 2015, Google Inc.
-# All rights reserved.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are
-# met:
-#
-#     * Redistributions of source code must retain the above copyright
-# notice, this list of conditions and the following disclaimer.
-#     * Redistributions in binary form must reproduce the above
-# copyright notice, this list of conditions and the following disclaimer
-# in the documentation and/or other materials provided with the
-# distribution.
-#     * Neither the name of Google Inc. nor the names of its
-# contributors may be used to endorse or promote products derived from
-# this software without specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-
-set -e
-
-if [ "x$TEST" = "x" ] ; then
-  TEST=false
-fi
-
-
-cd `dirname $0`/../..
-mako_renderer=tools/buildgen/mako_renderer.py
-
-if [ "x$TEST" != "x" ] ; then
-  tools/buildgen/build-cleaner.py build.yaml
-fi
-
-. tools/buildgen/generate_build_additions.sh
-
-global_plugins=`find ./tools/buildgen/plugins -name '*.py' |
-  sort | grep -v __init__ | awk ' { printf "-p %s ", $0 } '`
-
-for dir in . ; do
-  local_plugins=`find $dir/templates -name '*.py' |
-    sort | grep -v __init__ | awk ' { printf "-p %s ", $0 } '`
-
-  plugins="$global_plugins $local_plugins"
-
-  find -L $dir/templates -type f -and -name *.template | while read file ; do
-    out=${dir}/${file#$dir/templates/}  # strip templates dir prefix
-    out=${out%.*}  # strip template extension
-    echo "generating file: $out"
-    yaml_files="build.yaml $gen_build_files"
-    data=`for i in $yaml_files ; do echo $i ; done | awk ' { printf "-d %s ", $0 } '`
-    if [ "x$TEST" = "xtrue" ] ; then
-      actual_out=$out
-      out=`mktemp /tmp/gentXXXXXX`
-    fi
-    mkdir -p `dirname $out`  # make sure dest directory exist
-    $mako_renderer $plugins $data -o $out $file
-    if [ "x$TEST" = "xtrue" ] ; then
-      diff -q $out $actual_out
-      rm $out
-    fi
-  done
-done
-
-rm $gen_build_files
diff --git a/tools/dockerfile/interoptest/grpc_interop_http2/Dockerfile b/tools/dockerfile/interoptest/grpc_interop_http2/Dockerfile
index 03ff179f71..05e963d1e6 100644
--- a/tools/dockerfile/interoptest/grpc_interop_http2/Dockerfile
+++ b/tools/dockerfile/interoptest/grpc_interop_http2/Dockerfile
@@ -46,7 +46,6 @@ RUN apt-get update && apt-get install -y \
 RUN pip install pip --upgrade
 RUN pip install virtualenv
 RUN pip install futures==2.2.0 enum34==1.0.4 protobuf==3.0.0a2 six==1.10.0
-RUN pip install twisted h2
 
 # Define the default command.
 CMD ["bash"]
diff --git a/vsprojects/README.md b/vsprojects/README.md
index afd6430bfe..f1663d2548 100644
--- a/vsprojects/README.md
+++ b/vsprojects/README.md
@@ -1,8 +1,8 @@
-This directory contains MS Visual Studio project & solution files.
+#Pre-generated MS Visual Studio project & solution files
 
-#Supported Visual Studio versions
-
-Currently supported versions are Visual Studio 2013 (our primary focus).
+Versions 2013 and 2015 are both supported. You can use [their respective
+community
+editions](https://www.visualstudio.com/en-us/downloads/download-visual-studio-vs.aspx).
 
 #Building
 We are using [NuGet](http://www.nuget.org) to pull zlib and openssl dependencies.
@@ -16,8 +16,8 @@ download nuget.exe from the web and manually restore the NuGet packages.
 ```
 
 After that, you can build the solution using one of these options:
-1. open `grpc.sln` with Visual Studio and hit "Build".
-2. build from commandline using `msbuild grpc.sln /p:Configuration=Debug`
+  1. open `grpc.sln` with Visual Studio and hit "Build".
+  2. build from commandline using `msbuild grpc.sln /p:Configuration=Debug`
 
 #C/C++ Test Dependencies
    * gtest isn't available as a git repo like the other dependencies.  download it and add it to `/third_party/gtest/` (the folder will end up with `/build-aux/`, `/cmake/`, `/codegear/`, etc. folders in it).  
@@ -36,73 +36,18 @@ After that, you can build the solution using one of these options:
    * install [NuGet](http://www.nuget.org)
     * nuget should automatically bring in built versions of zlib and openssl when building grpc.sln (the versions in `/third_party/` are not used).  If it doesn't work use `tools->nuget...->manage...`.  The packages are put in `/vsprojects/packages/`
 
-#C/C++ Test Solution/Project Build Steps
-   * A basic git version of grpc only has templates for non-test items.  This checklist adds test items to grpc.sln and makes individual vs projects for them
-   * set up dependencies (above)
-   * add `"debug": true,` to the top of build.json.  This is the base file for all build tracking, see [templates](https://github.com/grpc/grpc/tree/master/templates) for more information
-    * `"debug": true,` gets picked up by `/tools/buildgen/plugins/generate_vsprojects.py`.  It tells the script to add visual studio GUIDs to all projects.  Otherwise only the projects that already have GUIDs in build.json will be built
-   * run `/templates/vsprojects/generate_debug_projects.sh` to make debug templates/projects.  This runs a regular visual studio buildgen process, which creates the `.sln` file with all of the new debug projects, then uses git diff to find the new project names from the `.sln` that need templates added.  It builds the new templates based on the diff, then re-runs the visual studio buildgen, which builds the vs projects for each of the new debug targets
-    * copy over the `/vsprojects/` folder to your windows build setup (assuming this was built on linux in order to have easy access to python/mako and shell scripts)
-   * run `/templates/vsprojects/build_test_protos.sh`
-    * this builds all `.proto` files in `/test/` in-place.  there might be a better place to put them that mirrors what happens in the linux build process (todo)
-    * each `.proto` file gets built into a `.grpc.pb.cc`, .`grpc.pb.h`, `.pb.cc`, and `.pb.h`.  These are included in each test project in lieu of the `.proto` includes specified in `build.json`.  This substitution is done by `/templates/vsprojects/vcxproj_defs.include`
-    * copy over the `/test/` folder in order to get the new files (assuming this was built on linux in order to have an easy protobuf+grpc plugin installation)
-
-#Making and running tests with `/tools/run_tests/run_tests.py` or `/vsprojects/make.bat`
-`run_tests.py` and `make.bat` both rely on `/vsprojects/grpc.mak`, an NMAKE script that includes C/C++ tests in addition to the base grpc projects.  It builds the base projects by calling grpc.sln, but most things are built with a command line similar to a makefile workflow.
-
- arguments for `/vsprojects/make.bat`:
-
- * no options or `all` or `buildtests`: builds all tests
- * `buildtests_c`: just c tests
- * `buildtests_cxx`: just c++ tests
- * names of individual tests: just those tests (example: `make.bat gpr_string_test`)
-
-using `run_tests.py` on windows:
-
- * when `run_tests.py` detects that it's running on windows it calls `make.bat` to build the tests and expects to find tests in `/vsprojects/test_bins/`
-
-`run_tests.py` options:
-
- * `run_tests.py --help`
- * `run_tests.py -l c`: run c language tests
- * `run_tests.py -l c++`: run c++ language tests
- * note: `run_tests.py` doesn't normally show build steps, so if a build fails it is best to fall back to `make.bat`
- * if `make.bat` fails, it might be easier to open up the `.sln` file in the visual studio gui (see above for how to build the test projects) and build the offending test from its project file.  The `.mak` and project file templates are slightly different, so it's possible that a project will build one way and not another.  Please report this if it happens.
-
-It can be helpful to disable the firewall when running tests so that 400 connection warnings don't pop up.
-
-Individual tests can be run by directly running the executable in `/vsprojects/run_tests/` (this is `/bins/opt/` on linux).  Many C tests have no output; they either pass or fail internally and communicate this with their exit code (`0=pass`, `nonzero=fail`)
-
-`run_tests.py` will fail if it can't build something, so not-building tests are disabled with a "platforms = posix" note in build.json.  The buildgen tools will not add a test to a windows build unless it is marked "windows" or has no platforms identified.  As tests are ported they will get this mark removed.
-
 # Building protoc plugins
 For generating service stub code, gRPC relies on plugins for `protoc` (the protocol buffer compiler). The solution `grpc_protoc_plugins.sln` allows you to build
 Windows .exe binaries of gRPC protoc plugins.
 
-1. Follow instructions in `third_party\protobuf\cmake\README.md` to create Visual Studio 2013 projects for protobuf.
-```
-$ cd third_party/protobuf/cmake
-$ mkdir build & cd build
-$ mkdir solution & cd solution
-$ cmake -G "Visual Studio 12 2013" -Dprotobuf_BUILD_TESTS=OFF ../..
-```
-
-2. Open solution `third_party\protobuf\cmake\build\solution\protobuf.sln` and build it in Release mode. That will build libraries `libprotobuf.lib` and `libprotoc.lib` needed for the next step.
+- Follow instructions in `third_party\protobuf\cmake\README.md` to create Visual Studio 2013 projects for protobuf.
+  ```
+  $ cd third_party/protobuf/cmake
+  $ mkdir build & cd build
+  $ mkdir solution & cd solution
+  $ cmake -G "Visual Studio 12 2013" -Dprotobuf_BUILD_TESTS=OFF ../..
+  ```
 
-3. Open solution `vsprojects\grpc_protoc_plugins.sln` and build it in Release mode. As a result, you should obtain a set of gRPC protoc plugin binaries (`grpc_cpp_plugin.exe`, `grpc_csharp_plugin.exe`, ...)
+- Open solution `third_party\protobuf\cmake\build\solution\protobuf.sln` and build it in Release mode. That will build libraries `libprotobuf.lib` and `libprotoc.lib` needed for the next step.
 
-#Building using CMake (with BoringSSL)
-1. Install [Active State Perl](http://www.activestate.com/activeperl/) (`choco install activeperl`)
-2. Install [Ninja](https://ninja-build.org/) (`choco install ninja`)
-2. Install [Go](https://golang.org/dl/) (`choco install golang`)
-3. Install [yasm](http://yasm.tortall.net/) and add it to `PATH` (`choco install yasm`)
-4. Update boringssl sumbodule to `master`
-5. Run this commads in grpc directory:
-```
-> md .build
-> cd .build
-> call "%VS140COMNTOOLS%..\..\VC\vcvarsall.bat" x64
-> cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release
-> cmake --build .
-```
+- Open solution `vsprojects\grpc_protoc_plugins.sln` and build it in Release mode. As a result, you should obtain a set of gRPC protoc plugin binaries (`grpc_cpp_plugin.exe`, `grpc_csharp_plugin.exe`, ...)