Merge 3.2.x branch into master (#2648)

* Down-integrate internal changes to github.

* Update conformance test failure list.

* Explicitly import used class in nano test to avoid random test fail.

* Update _GNUC_VER to use the correct implementation of atomic operation
on Mac.

* maps_test.js: check whether Symbol is defined before using it (#2524)

Symbol is not yet available on older versions of Node.js and so this
test fails with them. This change just directly checks whether Symbol is
available before we try to use it.

* Added well_known_types_embed.cc to CLEANFILES so that it gets cleaned up

* Updated Makefile.am to fix out-of-tree builds

* Added Bazel genrule for generating well_known_types_embed.cc

In pull request #2517 I made this change for the CMake and autotools
builds but forgot to do it for the Bazel build.

* Update _GNUC_VER to use the correct implementation of atomic operation on Mac.

* Add new js file in extra dist.

* Bump version number to 3.2.0

* Fixed issue with autoloading - Invalid paths (#2538)

* PHP fix int64 decoding (#2516)

* fix int64 decoding

* fix int64 decoding + tests

* Fix int64 decoding on 32-bit machines.

* Fix warning in compiler/js/embed.cc

embed.cc: In function ‘std::string CEscape(const string&)’:
embed.cc:51:32: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i = 0; i < str.size(); ++i) {
                                ^

* Fix include in auto-generated well_known_types_embed.cc

Restore include style fix (e3da722) that has been trampled by
auto-generation of well_known_types_embed.cc

* Fixed cross compilations with the Autotools build

Pull request #2517 caused cross compilations to start failing, because
the js_embed binary was being built to run on the target platform
instead of on the build machine. This change updates the Autotools build
to use the AX_PROG_CXX_FOR_BUILD macro to find a suitable compiler for
the build machine and always use that when building js_embed.

* Minor fix for autocreated object repeated fields and maps.

- If setting/clearing a repeated field/map that was objects, check the class
  before checking the autocreator.
- Just to be paranoid, don’t mutate within copy/mutableCopy for the autocreated
  classes to ensure there is less chance of issues if someone does something
  really crazy threading wise.
- Some more tests for the internal AutocreatedArray/AutocreatedDictionary
  classes to ensure things are working as expected.
- Add Xcode 8.2 to the full_mac_build.sh supported list.

* Fix generation of extending nested messages in JavaScript (#2439)

* Fix generation of extending nested messages in JavaScript

* Added missing test8.proto to build

* Fix generated code when there is no namespace but there is enum definition.

* Decoding unknown field should succeed.

* Add embed.cc in src/Makefile.am to fix dist check.

* Fixed "make distcheck" for the Autotools build

To make the test pass I needed to fix out-of-tree builds and update
EXTRA_DIST and CLEANFILES.

* Remove redundent embed.cc from src/Makefile.am

* Update version number to 3.2.0-rc.1 (#2578)

* Change protoc-artifacts version to 3.2.0-rc.1

* Update version number to 3.2.0rc2

* Update change logs for 3.2.0 release.

* Update php README

* Update upb, fixes some bugs (including a hash table problem). (#2611)

* Update upb, fixes some bugs (including a hash table problem).

* Ruby: added a test for the previous hash table corruption.

Verified that this triggers the bug in the currently released
version.

* Ruby: bugfix for SEGV.

* Ruby: removed old code for dup'ing defs.

* Reverting deployment target to 7.0 (#2618)

The Protobuf library doesn’t require the 7.1 deployment target so
reverting it back to 7.0

* Fix typo that breaks builds on big-endian (#2632)

* Bump version number to 3.2.0

9 files changed, 39 insertions, 49 deletions

diff --git a/src/google/protobuf/util/field_mask_util.cc b/src/google/protobuf/util/field_mask_util.cc
index 07550e6d..85cecec5 100644
--- a/src/google/protobuf/util/field_mask_util.cc
+++ b/src/google/protobuf/util/field_mask_util.cc
@@ -463,7 +463,8 @@ void FieldMaskTree::TrimMessage(const Node* node, Message* message) {
   const int32 field_count = descriptor->field_count();
   for (int index = 0; index < field_count; ++index) {
     const FieldDescriptor* field = descriptor->field(index);
-    map<string, Node*>::const_iterator it = node->children.find(field->name());
+    std::map<string, Node*>::const_iterator it =
+        node->children.find(field->name());
     if (it == node->children.end()) {
       reflection->ClearField(message, field);
     } else {
diff --git a/src/google/protobuf/util/internal/json_escaping.cc b/src/google/protobuf/util/internal/json_escaping.cc
index 06d2791b..47e4dd6d 100644
--- a/src/google/protobuf/util/internal/json_escaping.cc
+++ b/src/google/protobuf/util/internal/json_escaping.cc
@@ -336,19 +336,19 @@ StringPiece EscapeCodePoint(uint32 cp, char* buffer, bool force_output) {
     cp >>= 6;
     if (cp <= 0x1f) {
       buffer[4] = cp | 0xc0;
-      sp.set(buffer + 4, 2);
+      sp = StringPiece(buffer + 4, 2);
       return sp;
     }
     buffer[4] = (cp & 0x3f) | 0x80;
     cp >>= 6;
     if (cp <= 0x0f) {
       buffer[3] = cp | 0xe0;
-      sp.set(buffer + 3, 3);
+      sp = StringPiece(buffer + 3, 3);
       return sp;
     }
     buffer[3] = (cp & 0x3f) | 0x80;
     buffer[2] = ((cp >> 6) & 0x07) | 0xf0;
-    sp.set(buffer + 2, 4);
+    sp = StringPiece(buffer + 2, 4);
   }
   return sp;
 }
diff --git a/src/google/protobuf/util/internal/json_stream_parser.cc b/src/google/protobuf/util/internal/json_stream_parser.cc
index 2af4ad90..b38030c3 100644
--- a/src/google/protobuf/util/internal/json_stream_parser.cc
+++ b/src/google/protobuf/util/internal/json_stream_parser.cc
@@ -56,6 +56,7 @@ namespace util {
 // this file.
 using util::Status;
 namespace error {
+using util::error::CANCELLED;
 using util::error::INTERNAL;
 using util::error::INVALID_ARGUMENT;
 }  // namespace error
@@ -246,7 +247,7 @@ util::Status JsonStreamParser::RunParser() {
     }
     if (!result.ok()) {
       // If we were cancelled, save our state and try again later.
-      if (!finishing_ && result == util::Status::CANCELLED) {
+      if (!finishing_ && result == util::Status(error::CANCELLED, "")) {
         stack_.push(type);
         // If we have a key we still need to render, make sure to save off the
         // contents in our own storage.
@@ -290,7 +291,7 @@ util::Status JsonStreamParser::ParseValue(TokenType type) {
       // don't know if the next char would be e, completing it, or something
       // else, making it invalid.
       if (!finishing_ && p_.length() < false_len) {
-        return util::Status::CANCELLED;
+        return util::Status(error::CANCELLED, "");
       }
       return ReportFailure("Unexpected token.");
     }
@@ -328,7 +329,7 @@ util::Status JsonStreamParser::ParseStringHelper() {
       // depending on if we expect more data later.
       if (p_.length() == 1) {
         if (!finishing_) {
-          return util::Status::CANCELLED;
+          return util::Status(error::CANCELLED, "");
         }
         return ReportFailure("Closing quote expected in string.");
       }
@@ -396,7 +397,7 @@ util::Status JsonStreamParser::ParseStringHelper() {
   }
   // If we didn't find the closing quote but we expect more data, cancel for now
   if (!finishing_) {
-    return util::Status::CANCELLED;
+    return util::Status(error::CANCELLED, "");
   }
   // End of string reached without a closing quote, report an error.
   string_open_ = 0;
@@ -413,7 +414,7 @@ util::Status JsonStreamParser::ParseStringHelper() {
 util::Status JsonStreamParser::ParseUnicodeEscape() {
   if (p_.length() < kUnicodeEscapedLength) {
     if (!finishing_) {
-      return util::Status::CANCELLED;
+      return util::Status(error::CANCELLED, "");
     }
     return ReportFailure("Illegal hex string.");
   }
@@ -430,7 +431,7 @@ util::Status JsonStreamParser::ParseUnicodeEscape() {
       code <= JsonEscaping::kMaxHighSurrogate) {
     if (p_.length() < 2 * kUnicodeEscapedLength) {
       if (!finishing_) {
-        return util::Status::CANCELLED;
+        return util::Status(error::CANCELLED, "");
       }
       if (!coerce_to_utf8_) {
         return ReportFailure("Missing low surrogate.");
@@ -524,7 +525,7 @@ util::Status JsonStreamParser::ParseNumberHelper(NumberResult* result) {
   // If the entire input is a valid number, and we may have more content in the
   // future, we abort for now and resume when we know more.
   if (index == length && !finishing_) {
-    return util::Status::CANCELLED;
+    return util::Status(error::CANCELLED, "");
   }
 
   // Create a string containing just the number, so we can use safe_strtoX
@@ -679,7 +680,7 @@ util::Status JsonStreamParser::ParseArrayValue(TokenType type) {
   // empty-null array value is relying on this ARRAY_MID token.
   stack_.push(ARRAY_MID);
   util::Status result = ParseValue(type);
-  if (result == util::Status::CANCELLED) {
+  if (result == util::Status(error::CANCELLED, "")) {
     // If we were cancelled, pop back off the ARRAY_MID so we don't try to
     // push it on again when we try over.
     stack_.pop();
@@ -758,7 +759,7 @@ util::Status JsonStreamParser::ReportFailure(StringPiece message) {
 util::Status JsonStreamParser::ReportUnknown(StringPiece message) {
   // If we aren't finishing the parse, cancel parsing and try later.
   if (!finishing_) {
-    return util::Status::CANCELLED;
+    return util::Status(error::CANCELLED, "");
   }
   if (p_.empty()) {
     return ReportFailure(StrCat("Unexpected end of string. ", message));
@@ -788,7 +789,7 @@ util::Status JsonStreamParser::ParseKey() {
   // we can't know if the key was complete or not.
   if (!finishing_ && p_.empty()) {
     p_ = original;
-    return util::Status::CANCELLED;
+    return util::Status(error::CANCELLED, "");
   }
   // Since we aren't using the key storage, clear it out.
   key_storage_.clear();
diff --git a/src/google/protobuf/util/internal/protostream_objectsource.cc b/src/google/protobuf/util/internal/protostream_objectsource.cc
index a970dc12..3591febf 100644
--- a/src/google/protobuf/util/internal/protostream_objectsource.cc
+++ b/src/google/protobuf/util/internal/protostream_objectsource.cc
@@ -1024,15 +1024,8 @@ bool ProtoStreamObjectSource::IsMap(
     const google::protobuf::Field& field) const {
   const google::protobuf::Type* field_type =
       typeinfo_->GetTypeByTypeUrl(field.type_url());
-
-  // TODO(xiaofeng): Unify option names.
   return field.kind() == google::protobuf::Field_Kind_TYPE_MESSAGE &&
-         (GetBoolOptionOrDefault(field_type->options(),
-                                 "google.protobuf.MessageOptions.map_entry",
-                                 false) ||
-          GetBoolOptionOrDefault(field_type->options(), "map_entry", false) ||
-          GetBoolOptionOrDefault(field_type->options(),
-                                 "proto2.MessageOptions.map_entry", false));
+         google::protobuf::util::converter::IsMap(field, *field_type);
 }
 
 std::pair<int64, int32> ProtoStreamObjectSource::ReadSecondsAndNanos(
@@ -1128,4 +1121,3 @@ const string FormatNanos(uint32 nanos, bool with_trailing_zeros) {
 }  // namespace util
 }  // namespace protobuf
 }  // namespace google
-
diff --git a/src/google/protobuf/util/internal/protostream_objectsource_test.cc b/src/google/protobuf/util/internal/protostream_objectsource_test.cc
index bf29c17a..1286bdb9 100644
--- a/src/google/protobuf/util/internal/protostream_objectsource_test.cc
+++ b/src/google/protobuf/util/internal/protostream_objectsource_test.cc
@@ -114,7 +114,7 @@ class ProtostreamObjectSourceTest
   }
 
   Status ExecuteTest(const Message& msg, const Descriptor* descriptor) {
-    ostringstream oss;
+    std::ostringstream oss;
     msg.SerializePartialToOstream(&oss);
     string proto = oss.str();
     ArrayInputStream arr_stream(proto.data(), proto.size());
diff --git a/src/google/protobuf/util/internal/protostream_objectwriter.cc b/src/google/protobuf/util/internal/protostream_objectwriter.cc
index c37e671c..6c9bc30e 100644
--- a/src/google/protobuf/util/internal/protostream_objectwriter.cc
+++ b/src/google/protobuf/util/internal/protostream_objectwriter.cc
@@ -1238,13 +1238,7 @@ bool ProtoStreamObjectWriter::IsMap(const google::protobuf::Field& field) {
   const google::protobuf::Type* field_type =
       typeinfo()->GetTypeByTypeUrl(field.type_url());
 
-  // TODO(xiaofeng): Unify option names.
-  return GetBoolOptionOrDefault(field_type->options(),
-                                "google.protobuf.MessageOptions.map_entry",
-                                false) ||
-         GetBoolOptionOrDefault(field_type->options(), "map_entry", false) ||
-         GetBoolOptionOrDefault(field_type->options(),
-                                "proto2.MessageOptions.map_entry", false);
+  return google::protobuf::util::converter::IsMap(field, *field_type);
 }
 
 bool ProtoStreamObjectWriter::IsAny(const google::protobuf::Field& field) {
@@ -1269,4 +1263,3 @@ bool ProtoStreamObjectWriter::IsStructListValue(
 }  // namespace util
 }  // namespace protobuf
 }  // namespace google
-
diff --git a/src/google/protobuf/util/internal/utility.cc b/src/google/protobuf/util/internal/utility.cc
index 05759e85..6daf24eb 100644
--- a/src/google/protobuf/util/internal/utility.cc
+++ b/src/google/protobuf/util/internal/utility.cc
@@ -356,23 +356,25 @@ bool IsValidBoolString(const string& bool_string) {
 
 bool IsMap(const google::protobuf::Field& field,
            const google::protobuf::Type& type) {
-  return (
-      field.cardinality() ==
-          google::protobuf::Field_Cardinality_CARDINALITY_REPEATED &&
-      (GetBoolOptionOrDefault(
-           type.options(), "google.protobuf.MessageOptions.map_entry", false) ||
-       GetBoolOptionOrDefault(type.options(), "proto2.MessageOptions.map_entry",
-                              false)));
+  return field.cardinality() ==
+             google::protobuf::Field_Cardinality_CARDINALITY_REPEATED &&
+         (GetBoolOptionOrDefault(type.options(), "map_entry", false) ||
+          GetBoolOptionOrDefault(type.options(),
+                                 "google.protobuf.MessageOptions.map_entry", false) ||
+          GetBoolOptionOrDefault(type.options(),
+                                 "google.protobuf.MessageOptions.map_entry",
+                                 false));
 }
 
 bool IsMessageSetWireFormat(const google::protobuf::Type& type) {
-  return (
-      GetBoolOptionOrDefault(
-          type.options(),
-          "google.protobuf.MessageOptions.message_set_wire_format", false) ||
-      GetBoolOptionOrDefault(type.options(),
-                             "proto2.MessageOptions.message_set_wire_format",
-                             false));
+  return GetBoolOptionOrDefault(type.options(), "message_set_wire_format",
+                                false) ||
+         GetBoolOptionOrDefault(type.options(),
+                                "google.protobuf.MessageOptions.message_set_wire_format",
+                                false) ||
+         GetBoolOptionOrDefault(
+             type.options(),
+             "google.protobuf.MessageOptions.message_set_wire_format", false);
 }
 
 string DoubleAsString(double value) {
@@ -412,4 +414,3 @@ bool SafeStrToFloat(StringPiece str, float* value) {
 }  // namespace util
 }  // namespace protobuf
 }  // namespace google
-
diff --git a/src/google/protobuf/util/json_format_proto3.proto b/src/google/protobuf/util/json_format_proto3.proto
index 3835b30e..8a0441c8 100644
--- a/src/google/protobuf/util/json_format_proto3.proto
+++ b/src/google/protobuf/util/json_format_proto3.proto
@@ -32,6 +32,8 @@ syntax = "proto3";
 
 package proto3;
 
+option java_package = "com.google.protobuf.util";
+option java_outer_classname = "JsonFormatProto3";
 
 import "google/protobuf/duration.proto";
 import "google/protobuf/timestamp.proto";
diff --git a/src/google/protobuf/util/time_util.h b/src/google/protobuf/util/time_util.h
index 432c9f0a..b8846935 100644
--- a/src/google/protobuf/util/time_util.h
+++ b/src/google/protobuf/util/time_util.h
@@ -237,7 +237,7 @@ inline Duration operator%(const Duration& d1, const Duration& d2) {
   return result %= d2;
 }
 
-inline ostream& operator<<(ostream& out, const Duration& d) {
+inline std::ostream& operator<<(std::ostream& out, const Duration& d) {
   out << google::protobuf::util::TimeUtil::ToString(d);
   return out;
 }
@@ -284,7 +284,7 @@ inline Timestamp operator-(const Timestamp& t, const Duration& d) {
 }
 LIBPROTOBUF_EXPORT Duration operator-(const Timestamp& t1, const Timestamp& t2);
 
-inline ostream& operator<<(ostream& out, const Timestamp& t) {
+inline std::ostream& operator<<(std::ostream& out, const Timestamp& t) {
   out << google::protobuf::util::TimeUtil::ToString(t);
   return out;
 }