aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/shell/integration/discard_graph_edges_test.sh
Commit message (Collapse)AuthorAge
* Remove support for --discard_actions_after_execution.Gravatar tomlu2018-06-18
| | | | | | | The memory savings from this flag are not worth the complexity, and it interferes with action restarting. RELNOTES: Remove support for --discard_actions_after_execution. PiperOrigin-RevId: 201077905
* Add option --experimental_discard_actions_after_execution in order to turn ↵Gravatar janakr2018-05-15
| | | | | | | | it off after a release. Discarding actions hasn't been shown to have a significant positive effect on heap memory usage, after careful research by mschaller@. It's holding back other projects (threading Fileset metadata through) and adding complexity, so it's time to kill it. Out of an abundance of caution, I'll keep actions in memory via a flag flip, then, if it sticks, I'll change the default, and then I'll unwire everything. PiperOrigin-RevId: 196682768
* bash: Add a toolchain for local Bash.Gravatar Laszlo Csomor2018-04-16
| | | | | | | | | | | | | | | | | | | Bazel automatically detects the local Bash and creates a custom toolchain rule for it. Later, rules that use Bash will require this toolchain and retrieve Bash's path from it instead of relying on hardcoded paths or the `--shell_executable` flag. See https://github.com/bazelbuild/bazel/issues/4319 Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9 Closes #4980. Change-Id: Ic2406a4da260b284e15852070d58472ca18340af PiperOrigin-RevId: 193022708
* Add option to optionally wipe state at the end of a build.Gravatar ccalvarin2018-01-22
| | | | | | | This will serve as an alternative to --batch, leaving behind a server without state from the previous build. RELNOTES: Introduces --[no]keep_state_after_build PiperOrigin-RevId: 182778500
* Add an additional attempt to get a heap histogram since jmap on Bazel CI ↵Gravatar janakr2018-01-16
| | | | | | appears to be flaky in connecting to the JVM. PiperOrigin-RevId: 182118581
* Disable test_actions_deleted_after_execution_nobatch_keep_analysisGravatar dslomov2018-01-15
| | | | | RELNOTES: None. PiperOrigin-RevId: 181971087
* Rename --keep_incrementality_data to --track_incremental_state.Gravatar ccalvarin2017-12-14
| | | | | | | | | New name clears the namespace a 2nd flag that will wipe the build graph after the build. The old name would be confusing as it could easily apply to that, and so needs to be more specifically just about tracking state in the first place. The new flag can be clearly separate and about keeping state after the build. Partial roll forward of https://github.com/bazelbuild/bazel/commit/9321316b34767b06c3071b2cf2a4de189874fcce, with fixes to documentation that are still relevant. RELNOTES: Rename --keep_incrementality_data to --track_incremental_state PiperOrigin-RevId: 179078292
* Automated rollback of commit 9321316b34767b06c3071b2cf2a4de189874fcce.Gravatar ccalvarin2017-12-11
| | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Design change, 2 boolean flags instead of 1 enum flag *** Original change description *** Add --incremental_state_retention_strategy This option is intended to replace some of the uses of --batch. It lets users specify that builds should not be incremental, and how eagerly to discard the state that is kept around for incrementality. Note that for both values discard_eargerly and keep_for_life_of_build, the build graph is kept around until the next build. This may change. Will add tests for keep_for_life_of_build in a later change, for now it will warn that that feature is experimen... *** ROLLBACK_OF=178661777 RELNOTES: None. PiperOrigin-RevId: 178681472
* Add --incremental_state_retention_strategyGravatar ccalvarin2017-12-11
| | | | | | | | | This option is intended to replace some of the uses of --batch. It lets users specify that builds should not be incremental, and how eagerly to discard the state that is kept around for incrementality. Note that for both values discard_eargerly and keep_for_life_of_build, the build graph is kept around until the next build. This may change. Will add tests for keep_for_life_of_build in a later change, for now it will warn that that feature is experimental. RELNOTES: --[no]keep_incrementality_data is gone, replaced by the enum-valued --incremental_state_retention_strategy PiperOrigin-RevId: 178661777
* RELNOTES: --keep_incrementality_data flag allows Bazel servers to be run in ↵Gravatar janakr2017-11-12
| | | | | | | | | | | | | | memory-saving non-incremental mode independent of --batch and --discard_analysis_cache. A command run with --nokeep_incrementality_data will discard data that would be needed for incremental builds. Subsequent commands can be sent to the same server, but they will not get the benefit of incrementality from this command. However, if --keep_incrementality_data is specified on a subsequent command, the commands after that will get the benefits of incrementality. There are two benefits to not being dependent on --batch. First, this allows Bazel servers to be run in extreme memory-saving mode without the startup penalties (JVM startup, JITting) that --batch execution imposes. Second, this allows Bazel developers to inspect the state of a Bazel server after an extreme memory-saving build. In order to avoid discarding data unnecessarily (for instance, on a "bazel info used-heap-size-after-gc" or "bazel dump --skyframe=summary") the actual resetting of the graph is done lazily, right before its use in SequencedSkyframeExecutor#sync. This is morally a partial rollback of https://github.com/bazelbuild/bazel/commit/98cd82cbdcac7c48164a611c5a9aa8fc2f1720ef. For now, our tests specify all of the flags. After this change sticks, I plan to get rid of the --batch flag from these tests, which should allow for some clean-ups. Eventually --batch and --discard_analysis_cache may not imply that we don't keep incremental state: we can require that it be specified explicitly. PiperOrigin-RevId: 175335075
* Fix a bug in which --experimental_post_build_query crashes if asked to run ↵Gravatar janakr2017-11-10
| | | | | | on a graph without edges. Now we fail gracefully. PiperOrigin-RevId: 175294923
* Don't require --keep_going to discard graph edges. It's unnecessary.Gravatar janakr2017-11-02
| | | | PiperOrigin-RevId: 174202685
* Extract test_actions_deleted_after_execution to a library for internal use.Gravatar janakr2017-10-27
| | | | PiperOrigin-RevId: 173607594
* Reset evaluator eagerly when incremental state changes. And add a regression ↵Gravatar janakr2017-10-24
| | | | | | test to make sure we are using the expected type of node entries when discarding/keeping graph edges. PiperOrigin-RevId: 173131307
* Unbreak discard_graph_edges_test.Gravatar philwo2017-09-12
| | | | PiperOrigin-RevId: 168220770
* Add a new toolchain type for c++. In order to do this, ↵Gravatar cpeyser2017-09-08
| | | | | | | | PlatformConfiguration is made a legal configuration fragment for every rule class. Add a default "dummy" c++ toolchain to prevent resolution errors when legacy toolchain selection logic is used. Add toolchain mocks to java and shell tests. PiperOrigin-RevId: 167901210
* Drop lower limit for RuleConfiguredTarget in object-counting test to account ↵Gravatar pcloudy2017-09-05
| | | | | | | | for Linux platforms. Fix discard_graph_edges_test.sh PiperOrigin-RevId: 167568133
* Automated rollback of commit 964f0a5d83fe7958236c9eb0be619f553d488748.Gravatar cpeyser2017-09-01
| | | | | | | | | | | | *** Reason for rollback *** Breaks rules_go CI *** Original change description *** Bazel c++ rules depend on a c++ toolchain. PiperOrigin-RevId: 167191667
* Bazel c++ rules depend on a c++ toolchain.Gravatar cpeyser2017-09-01
| | | | PiperOrigin-RevId: 167147239
* Automated rollback of commit 9d5bce66280094c0531ddfde2752e596d0d78c79.Gravatar plf2017-08-30
| | | | PiperOrigin-RevId: 166966182
* Add a new toolchain type for c++. In order to do this, ↵Gravatar cpeyser2017-08-29
| | | | | | | | PlatformConfiguration is made a legal configuration fragment for every rule class. Add a default "dummy" c++ toolchain to prevent resolution errors when legacy toolchain selection logic is used. Add toolchain mocks to java and shell tests. PiperOrigin-RevId: 166854893
* Automated rollback of commit 363413110f3a63a11d900864be1852adbaed2899.Gravatar cpeyser2017-08-29
| | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks //src/test/shell/bazel:bazel_bootstrap_distfile_test: INFO: You can skip this first step by providing a path to the bazel binary as second argument: INFO: ./compile.sh compile /path/to/bazel 🍃 Building Bazel from scratch...... 🍃 Building Bazel with Bazel. .WARNING: /tmp/bazel_cHivhPBc/out/external/bazel_tools/WORKSPACE:1: Workspace name in /tmp/bazel_cHivhPBc/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions. ERROR: in target '//external:cc_toolchain': error loading package '@local_config_cc//': Extension file not found. Unable to load file '@local_config_cc//:dummy_toolchain.bzl': file doesn't exist or isn't a file. INFO: Elapsed time: 3.343s ERROR: Could not build Bazel Found by git bisect. *** Original change description *** Add a new toolchain type for c++. In order to do this, PlatformConfiguration is made a legal configuration fragment for every rule class. Add a default "dummy" c++ toolchain to prevent resolution errors when legacy toolchain selection logic is used. Add toolchain mocks to java and shell tests. PiperOrigin-RevId: 166750885
* Add debugging to other flaky test case in discard_graph_edges_test. We ↵Gravatar janakr2017-08-29
| | | | | | actually don't need most of the information being logged here, but on the other hand, I have no reports of flakiness from the other test case after these log lines were added, so maybe it'll be a heisenbug and go away. PiperOrigin-RevId: 166731945
* Enforce that there should be no RuleConfiguredTarget objects on the heap ↵Gravatar janakr2017-08-28
| | | | | | during execution when using --discard_analysis_cache. PiperOrigin-RevId: 166611136
* Replace deprecated `set` constructor and its order names in integration testsGravatar vladmos2017-08-24
| | | | | | | The name `set` and order names "stable", "compile", "naive_link", and "link" are deprecated and will soon be removed from Blaze. PiperOrigin-RevId: 166341984
* Remove unused module-related optionsGravatar hlopko2017-07-28
| | | | | RELNOTES: None. PiperOrigin-RevId: 163338873
* Write directly to stdout when debugging in test: otherwise test log can get ↵Gravatar janakr2017-07-24
| | | | | | overwritten, losing info (http://ci.bazel.io/job/bazel-tests/899/BAZEL_VERSION=latest,PLATFORM_NAME=ubuntu_16.04-x86_64/consoleFull). PiperOrigin-RevId: 162764897
* Be more precise in which values we're looking for in the histogram: just ↵Gravatar janakr2017-07-20
| | | | | | GlobValue objects, not any others. PiperOrigin-RevId: 162533962
* Add debugging for flaky test: maybe fifo problem.Gravatar Janak Ramakrishnan2017-07-07
| | | | | Change-Id: I370310592a96889b4b69965ee0ba325337784b71 PiperOrigin-RevId: 161138809
* Migrate tests to ctx.actions.run/run_shell.Gravatar dslomov2017-07-05
| | | | | RELNOTES: None. PiperOrigin-RevId: 160817326
* Introduce ctx.actions.write in place of ctx.file_action.Gravatar dslomov2017-06-30
| | | | | RELNOTES: None. PiperOrigin-RevId: 160630261
* Add 'ctx.actions' and implement 'ctx.action.declare_file'.Gravatar dslomov2017-06-27
| | | | | RELNOTES: None. PiperOrigin-RevId: 160264501
* Disable test_packages_cleared because it is flaky on CI.Gravatar jcater2017-06-09
| | | | | | Internal bug has been filed. PiperOrigin-RevId: 158529189
* Automated g4 rollback of commit 6879d7ceff0e118fdecb0cabe5134979030b7cb8.Gravatar Googler2017-05-11
| | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Fixes memory issue that caused this CL to be rolled back. *** Original change description *** Automated g4 rollback of commit cbbb423663b154d82e3dfa5e9a56839583987999. *** Reason for rollback *** Need to roll this back as part of http://b/38171368 *** Original change description *** RELNOTES: Effectively remove sysroot from CppConfiguration and allow it to use select statements. PiperOrigin-RevId: 155651879
* Automated g4 rollback of commit cbbb423663b154d82e3dfa5e9a56839583987999.Gravatar Googler2017-05-10
| | | | | | | | | | | *** Reason for rollback *** Need to roll this back as part of http://b/38171368 *** Original change description *** RELNOTES: Effectively remove sysroot from CppConfiguration and allow it to use select statements. PiperOrigin-RevId: 155547813
* RELNOTES: Effectively remove sysroot from CppConfiguration and allow it to ↵Gravatar Googler2017-05-09
| | | | | | use select statements. PiperOrigin-RevId: 155480011
* test_action_conflict no longer relies on cc_binary internalsGravatar cparsons2017-04-12
| | | | | | | This test previously depended on cc_binary generation of .pic files, which will not be applicable for osx architectures. RELNOTES: None. PiperOrigin-RevId: 152822069
* Remove static declaration of apple split transitions, as this is no longer ↵Gravatar cparsons2017-04-10
| | | | | | | necessary with dynamic configurations RELNOTES: None. PiperOrigin-RevId: 152515642
* When tracking the critical path, if not keeping incremental state, don't ↵Gravatar janakr2017-04-06
| | | | | | | | keep references to actions indefinitely. Instead, once an action is finished executing, keep just some metadata about it. This allows actions to be unconditionally dropped when running with --batch, --discard_analysis_cache, and --keep_going, even if profiling is enabled. The additional fields here add between 8 and 12 bytes per component, and we have one component per action. This additional penalty is only incurred when we are already saving memory, so I think it's ok. The full penalty will be realized only towards the end of the build, when every action has started executing at least once. Users can still specify --noexperimental_enable_critical_path_profiling if they want to squeeze even more memory out. PiperOrigin-RevId: 152328870
* If --batch, --keep_going, --discard_analysis_cache, and the new ↵Gravatar janakr2017-03-31
| | | | | | | | | | | | | | | --noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory. Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata. Some actions unfortunately are needed post-execution, and so we special-case them. Also includes dependency change with description: Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys. This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds. PiperOrigin-RevId: 151756383
* Add option to disable critical path computation in Bazel. While this ↵Gravatar janakr2017-03-31
| | | | | | | | | | information is useful, the critical path computer retains references to objects that could otherwise be cleared to save memory. This change is probably not worth submitting on its own -- the benefit it provides is too slight. But my follow-up change unknown commit needs this option to be effective -- the critical path currently hangs on to references to every action in the graph, so we can't drop references to actions if it's enabled. The critical path could probably be reworked in the future to not hang onto those references. PiperOrigin-RevId: 151747605
* Drop loading-phase values if --discard_analysis_cache is true and we're not ↵Gravatar janakr2017-03-31
| | | | | | keeping incremental state. PiperOrigin-RevId: 151639711
* Clear AspectValues when discarding analysis cache, along with ↵Gravatar janakr2017-03-29
| | | | | | | | ConfiguredTargetValues. Also clear transitive packages for both, even for top-level targets. This is not expected to save significant memory, but is expected to reduce the number of references to Packages, allowing them to be dropped more easily when discarding analysis cache and running in batch mode. PiperOrigin-RevId: 151508877
* Create a proper wrapper script for executing "bazel" in the integration tests.Gravatar Luis Fernando Pino Duque2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently a call to "bazel" in an integration test means calling a (quite hidden) function in test-setup.sh which actually calls "$bazel" defined in "shell/bazel/testenv.sh" which is equal to "$(rlocation io_bazel/src/bazel)". This is extremely confusing and error prone. The new mechanism is to add a wrapper script to shell/bin called bazel and export this directory to the PATH. Moreover, not every test loads the same test environment, for instance consider how bazel_query_test loads the test environment: - Load shell/integration/testenv.sh which loads, - shell/bazel/test-setup.sh which loads, - shell/bazel/testenv.sh which loads, - shell/unittest.bash which loads, - shell/testenv.sh Again this is error prone and specially hard to understand, in fact each test writer needs to decide which of these testenv to load. This change fixes all of this by having only one testenv.sh and summarizing the test setup in integration_test_setup.sh. Namely, for any new integration test, the developer needs to load integration_test_setup to get the environment set up including the unittest framework (also it helps to attract contributions). This change also allows to open sourcing client_sigint_test: Since bazel was a function client_sigint_test was using a wrong process id to interrupt the build. The problem is that $! returns bash's id instead of the id of the process running in the background when using a function instead of an executable. A few tests needed to be adapted to the new infrastructure. -- MOS_MIGRATED_REVID=136470360
* Open source discard_graph_edges_test.Gravatar John Cater2016-09-14
-- MOS_MIGRATED_REVID=133113800
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-27 22:03:05 +0000