- 12 May, 2023 2 commits
-
-
Извеков Василий Михайлович authored
-
Alisha Nanda authored
-
- 11 May, 2023 9 commits
-
-
Xuan Wang authored
Reverts grpc/grpc#32909 It's breaking some internal test (//net/grpc/python:internal_tests/unit/_default_reflection_test), revert for now to investigate.
-
Doug Fawley authored
-
Mark D. Roth authored
Also remove a check in the weighted_target LB policy that I somehow missed in #32932.
-
Thomas Zeugner authored
Fix #32899 - include all dependency-descriptors in the ServerReflectionResponse Using the C# Server-code as blue print: https://github.com/grpc/grpc-dotnet/blob/6bf44820a2ef6af5845830728cd4b3d1a8053ed4/src/Grpc.Reflection/ReflectionServiceImpl.cs#L148 see also: - The cpp-server implementation of this: https://github.com/grpc/grpc/blob/d299f5ecce9f8e9a8f35571f39bdcfdbb2106d36/src/cpp/ext/proto_server_reflection.cc#L208 - The go-server implementation of this: https://github.com/grpc/grpc-go/blob/2cd95c7514a3d02aa2d98591c013885cb44fbdeb/reflection/serverreflection.go#L184 `return s.fileDescWithDependencies(d.ParentFile(), sentFileDescriptors)` --------- Co-authored-by:
Xuan Wang <xuanwn@google.com>
-
Yash Tibrewal authored
Change - just make sure that before we verify the logging entries, we'll wait for the expected number of entries to show up. Logging_test has been recently reported as flaky. Sample failure - https://source.cloud.google.com/results/invocations/ba581ad8-b652-4b9d-af56-07593f5d2deb/targets/%2F%2Ftest%2Fcpp%2Fext%2Ffilters%2Flogging:logging_test@poller%3Depoll1/tests Failed to reproduce, but i have a suspicion that what's happening is that the logging for the half-close events on the server side can happen after the call has ended.. It is currently being logged after the server-trailer (which has the status).
-
AJ Heller authored
A known bug is being worked on. This will temporarily reduce flaky test noise.
-
Craig Tiller authored
Add a new binary that runs all core end2end tests in fuzzing mode. In this mode FuzzingEventEngine is substituted for the default event engine. This means that time is simulated, as is IO. The FEE gets control of callback delays also. In our tests the `Step()` function becomes, instead of a single call to `completion_queue_next`, a series of calls to that function and `FuzzingEventEngine::Tick`, driving forward the event loop until progress can be made. PR guide: --- **New binaries** `core_end2end_test_fuzzer` - the new fuzzer itself `seed_end2end_corpus` - a tool that produces an interesting seed corpus **Config changes for safe fuzzing** The implementation tries to use the config fuzzing work we've previously deployed in api_fuzzer to fuzz across experiments. Since some experiments are far too experimental to be safe in such fuzzing (and this will always be the case): - a new flag is added to experiments to opt-out of this fuzzing - a new hook is added to the config system to allow variables to re-write their inputs before setting them during the fuzz **Event manager/IO changes** Changes are made to the event engine shims so that tcp_server_posix can run with a non-FD carrying EventEngine. These are in my mind a bit clunky, but they work and they're in code that we expect to delete in the medium term, so I think overall the approach is good. **Changes to time** A small tweak is made to fix a bug initializing time for fuzzers in time.cc - we were previously failing to initialize `g_process_epoch_cycles` **Changes to `Crash`** A version that prints to stdio is added so that we can reliably print a crash from the fuzzer. **Changes to CqVerifier** Hooks are added to allow the top level loop to hook the verification functions with a function that steps time between CQ polls. **Changes to end2end fixtures** State machinery moves from the fixture to the test infra, to keep the customizations for fuzzing or not in one place. This means that fixtures are now just client/server factories, which is overall nice. It did necessitate moving some bespoke machinery into h2_ssl_cert_test.cc - this file is beginning to be problematic in borrowing parts but not all of the e2e test machinery. Some future PR needs to solve this. A cq arg is added to the Make functions since the cq is now owned by the test and not the fixture. **Changes to test registration** `TEST_P` is replaced by `CORE_END2END_TEST` and our own test registry is used as a first depot for test information. The gtest version of these tests: queries that registry to manually register tests with gtest. This ultimately changes the name of our tests again (I think for the last time) - the new names are shorter and more readable, so I don't count this as a regression. The fuzzer version of these tests: constructs a database of fuzzable tests that it can consult to look up a particular suite/test/config combination specified by the fuzzer to fuzz against. This gives us a single fuzzer that can test all 3k-ish fuzzing ready tests and cross polinate configuration between them. **Changes to test config** The zero size registry stuff was causing some problems with the event engine feature macros, so instead I've removed those and used GTEST_SKIP in the problematic tests. I think that's the approach we move towards in the future. **Which tests are included** Configs that are compatible - those that do not do fd manipulation directly (these are incompatible with FuzzingEventEngine), and those that do not join threads on their shutdown path (as these are incompatible with our cq wait methodology). Each we can talk about in the future - fd manipulation would be a significant expansion of FuzzingEventEngine, and is probably not worth it, however many uses of background threads now should probably evolve to be EventEngine::Run calls in the future, and then would be trivially enabled in the fuzzers. Some tests currently fail in the fuzzing environment, a `SKIP_IF_FUZZING` macro is used for these few to disable them if in the fuzzing environment. We'll burn these down in the future. **Changes to fuzzing_event_engine** Changes are made to time: an exponential sweep forward is used now - this catches small time precision things early, but makes decade long timers (we have them) able to be used right now. In the future we'll just skip time forward to the next scheduled timer, but that approach doesn't yet work due to legacy timer system interactions. Changes to port assignment: we ensure that ports are legal numbers before assigning them via `grpc_pick_port_or_die`. A race condition between time checking and io is fixed. --------- Co-authored-by:
ctiller <ctiller@users.noreply.github.com>
-
Hannah Shi authored
Added `//:gpr_platform` to cf_engine_test to fix build_cleaner check in the previous merge. More details in https://github.com/grpc/grpc/pull/33027
-
molto authored
Channel object has no method ``stop``, it supposed to be ``close`` here. --------- Co-authored-by:
Xuan Wang <xuanwn@google.com>
-
- 10 May, 2023 6 commits
-
-
Yingwei Fan authored
Co-authored-by:
Yash Tibrewal <yashkt@google.com> Co-authored-by:
Stanley Cheung <stanleycheung@google.com> Co-authored-by:
AJ Heller <hork@google.com> Co-authored-by:
Yijie Ma <yijiem.main@gmail.com> Co-authored-by:
apolcyn <apolcyn@google.com> Co-authored-by:
Jan Tattermusch <jtattermusch@google.com>
-
Mark D. Roth authored
Fixes #32967. Also fix incorrect defaults for `enforcementPercentage` fields.
-
Craig Tiller authored
Need to do the channelz bit prior to the finishing the op bit. <!-- If you know who should review your pull request, please assign it to that person, otherwise the pull request would get assigned randomly. If your pull request is for a specific language, please add the appropriate lang label. -->
-
Sergii Tkachenko authored
Resolve `TESTING_VERSION` to `dev-VERSION` when the job is initiated by a user, and not the CI. Override this behavior with setting `FORCE_TESTING_VERSION`. This solves the problem with the manual job runs executed against a WIP branch (f.e. a PR) overriding the tag of the CI-built image we use for daily testing. The `dev` and `dev-VERSION` "magic" values supported by the `--testing_version` flag: - `dev` and `dev-master` and treated as `master`: all `config.version_gte` checks resolve to `True`. - `dev-VERSION` is treated as `VERSION`: `dev-v1.55.x` is treated as simply `v1.55.x`. We do this so that when manually running jobs for old branches the feature skip check still works, and unsupported tests are skipped. This changes will take care of all langs/branches, no backports needed. ref b/256845629
-
AJ Heller authored
-
Benjamin Peterson authored
-
- 09 May, 2023 11 commits
-
-
Sergii Tkachenko authored
Since we use Logs Explorer.
-
AJ Heller authored
The `CanStartLotsOfClosures` test was sometimes taking over 60s to run ([example](https://source.cloud.google.com/results/invocations/d96c89f9-03f9-43fd-a729-d744d7499532/targets;query=thread_pool_test/%2F%2Ftest%2Fcore%2Fevent_engine:thread_pool_test@poller%3Depoll1/log)). More often than not, though, the test would take < 5s ([example](https://source.cloud.google.com/results/invocations/95d32b32-5df7-4dd4-a82c-1024869b09c8/targets;query=thread_pool_test/%2F%2Ftest%2Fcore%2Fevent_engine:thread_pool_test/log)). Both examples are from before the tests changed with the introduction of the work-stealing thread pool (https://github.com/grpc/grpc/commit/3fb738b9b1835b8b1faf08cfacfe0b516fb29e22). This PR reduces the closure count to 500k for the `CanStartLotsOfClosures` test, and changes the blocking-closure scale-test to exercise the work stealing implementation alone.
-
Mark D. Roth authored
Currently, we are not very consistent in what we assume the initial state of an LB policy will be and whether or not we assume that it will immediately report a new picker when it gets its initial address update; different parts of our code make different assumptions. This PR establishes the convention that LB policies will be assumed to start in state CONNECTING and will *not* be assumed to report a new picker immediately upon getting their initial address update, and we now assume that convention everywhere consistently. This is a preparatory step for changing policies like round_robin to delegate to pick_first, which I'm working on in #32692. As part of that change, we need pick_first to not report a connectivity state until it actually sees the connectivity state of the underlying subchannels, so that round_robin knows when to swap over to a new child list without reintroducing the problem fixed in #31939.
-
Esun Kim authored
Upgrading Protobuf and Upb to v23.0
-
Esun Kim authored
To fix this error ``` test/core/security/grpc_authorization_engine_test.cc:88:32: error: unknown type name 'Json'; did you mean 'experimental::Json'? ParseAuditLoggerConfig(const Json&) override { ^~~~ experimental::Json ```
-
Matthew Stevenson authored
This check compares the host portion of the target name to the authority header, but in common use cases (e.g. GCS) they may not coincide. Additionally, this check does not happen in the Go and Java ALTS stacks.
-
Eugene Ostroukhov authored
Part of the work needed for [gRFC A60](https://github.com/grpc/proposal/blob/master/A60-xds-stateful-session-affinity-weighted-clusters.md).
-
Mark D. Roth authored
This makes the JSON API visible as part of the C-core API, but in the `experimental` namespace. It will be used as part of various experimental APIs that we will be introducing in the near future, such as the audit logging API.
-
Ming-Chuan authored
WireWriter implementation schedules actions to be run by `ExecCtx`. We should flush pending actions before destructing `end2end_testing::g_transaction_processor`, which need to be alive to handle the scheduled actions. Otherwise, we get heap-use-after-free error because the testing fixture (`end2end_testing::g_transaction_processor`) is destructed before all the scheduled actions are run. This lowers end2end binder transport test failure rate from 0.23% to 0.15%, according to internal tool that runs the test for 15000 times under various configuration.
-
Mark D. Roth authored
This corresponds to two recent changes made to our internal implementation. See b/276292666 for details.
-
Craig Tiller authored
This file does not contain a shebang, and whenever I try and run it it wedges my console into some weird state. There's a .sh file with the same name that should be run instead. Remove the executable bit of the thing we shouldn't run directly so we, like, don't. <!-- If you know who should review your pull request, please assign it to that person, otherwise the pull request would get assigned randomly. If your pull request is for a specific language, please add the appropriate lang label. -->
-
- 08 May, 2023 5 commits
-
-
AJ Heller authored
`synchronized` is not a valid keyword. It's a shame this isn't reported anywhere I can see. See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows
-
Sergii Tkachenko authored
Previously the error message didn't provide much context, example: ```py Traceback (most recent call last): File "/tmpfs/tmp/tmp.BqlenMyXyk/grpc/tools/run_tests/xds_k8s_test_driver/tests/affinity_test.py", line 127, in test_affinity self.assertLen( AssertionError: [] has length of 0, expected 1. ``` ref b/279990584.
-
AJ Heller authored
This PR implements a work-stealing thread pool for use inside EventEngine implementations. Because of historical risks here, I've guarded the new implementation behind an experiment flag: `GRPC_EXPERIMENTS=work_stealing`. Current default behavior is the original thread pool implementation. Benchmarks look very promising: ``` bazel test \ --test_timeout=300 \ --config=opt -c opt \ --test_output=streamed \ --test_arg='--benchmark_format=csv' \ --test_arg='--benchmark_min_time=0.15' \ --test_arg='--benchmark_filter=_FanOut' \ --test_arg='--benchmark_repetitions=15' \ --test_arg='--benchmark_report_aggregates_only=true' \ test/cpp/microbenchmarks:bm_thread_pool ``` 2023-05-04: `bm_thread_pool` benchmark results on my local machine (64 core ThreadRipper PRO 3995WX, 256GB memory), comparing this PR to master:  2023-05-04: `bm_thread_pool` benchmark results in the Linux RBE environment (unsure of machine configuration, likely small), comparing this PR to master.  --------- Co-authored-by:
drfloob <drfloob@users.noreply.github.com>
-
Yijie Ma authored
One TXT lookup query can return multiple TXT records (see the following example). `EventEngine::DNSResolver` should return all of them to let the caller (e.g. `event_engine_client_channel_resolver`) decide which one they would use. ``` $ dig TXT wikipedia.org ; <<>> DiG 9.18.12-1+build1-Debian <<>> TXT wikipedia.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49626 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;wikipedia.org. IN TXT ;; ANSWER SECTION: wikipedia.org. 600 IN TXT "google-site-verification=AMHkgs-4ViEvIJf5znZle-BSE2EPNFqM1nDJGRyn2qk" wikipedia.org. 600 IN TXT "yandex-verification: 35c08d23099dc863" wikipedia.org. 600 IN TXT "v=spf1 include:wikimedia.org ~all" ``` Note that this change also deviates us from the iomgr's DNSResolver API which uses std::string as the result type. <!-- If you know who should review your pull request, please assign it to that person, otherwise the pull request would get assigned randomly. If your pull request is for a specific language, please add the appropriate lang label. -->
-
Matthew Stevenson authored
Having this logging will help debug a current failure.
-
- 05 May, 2023 5 commits
-
-
Xuan Wang authored
<!-- If you know who should review your pull request, please assign it to that person, otherwise the pull request would get assigned randomly. If your pull request is for a specific language, please add the appropriate lang label. --> --------- Co-authored-by:
Sergii Tkachenko <hi@sergii.org>
-
Easwar Swaminathan authored
I noticed that the [PR](https://github.com/grpc/grpc/pull/32683) to add v1.54.0 is still not merged. So, I added a line for that as well.
-
AJ Heller authored
Reverts grpc/grpc#32924. This breaks the build again, unfortunately. From `test/core/event_engine/cf:cf_engine_test`: ``` error: module .../grpc/test/core/event_engine/cf:cf_engine_test does not depend on a module exporting 'grpc/support/port_platform.h' ``` @sampajano I recommend looking into CI tests to catch iOS problems before merging. We can enable EventEngine experiments in the CI generally once this PR lands, but this broken test is not one of those experiments. A normal build should have caught this. cc @HannahShiSFB
-
Bulat Gaifullin authored
<!-- If you know who should review your pull request, please assign it to that person, otherwise the pull request would get assigned randomly. If your pull request is for a specific language, please add the appropriate lang label. -->
-
Hannah Shi authored
bazel build --config=macos --genrule_strategy=local --copt="-DGRPC_CFSTREAM=1" //test/cpp/end2end:cfstream_test succeeds Fixing failure described here: https://github.com/grpc/grpc/pull/32882#issuecomment-1512210309
-
- 04 May, 2023 2 commits
-
-
Mark D. Roth authored
Original was #33002, reverted in #33014. The second commit here adds a build visibility tag necessary to fix the internal build problems.
-
Sergii Tkachenko authored
The job run time was creeping to the 2h timeout. Let's bump it to 3h. Note that this is `master` branch, so it also includes the build time every time we commit to grpc/grpc. ref b/280784903
-