Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

PHPC-2261: Revamp Evergreen config #1473

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alcaeus merged 44 commits into mongodb:master from alcaeus:phpc-2201-2260-revamp-evergreen-config
Oct 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
a67441c
Remove unused functions and tasks
alcaeus Sep 18, 2023
dc6df70
Fix indentation
alcaeus Sep 18, 2023
780a9e4
POC: Use task group to test local deployments
alcaeus Sep 18, 2023
8f99925
Add make target to test without building
alcaeus Sep 19, 2023
16b8e44
Cache compiled build for later use
alcaeus Sep 19, 2023
7a4b177
Rename tasks for better filtering
alcaeus Sep 19, 2023
038ec06
Remove working dir upload as it's already uploaded along with the build
alcaeus Sep 19, 2023
d8dbc3d
Update tags in test tasks
alcaeus Sep 19, 2023
a131122
Create build variants for building driver
alcaeus Sep 19, 2023
94660d5
Fetch correct build file
alcaeus Sep 19, 2023
8a72a19
Extract configuration to separate files
alcaeus Sep 20, 2023
6815c3e
Add task group for builds
alcaeus Sep 20, 2023
137fa47
Add builds for different libmongoc versions
alcaeus Sep 20, 2023
dbda695
Test MongoDB 7.0 on Debian 11
alcaeus Sep 20, 2023
9376fb2
Move Atlas Connectivity and crypt_shared tests to new format
alcaeus Sep 20, 2023
c06b7e7
Move storage engine and libmongoc variants to new format
alcaeus Sep 20, 2023
74f3e04
Run smoketest after build
alcaeus Sep 20, 2023
9535c73
Extract generatable tasks
alcaeus Sep 22, 2023
4a20f4e
Remove obsolete file
alcaeus Sep 22, 2023
bebb155
Remove obsolete variant
alcaeus Sep 22, 2023
6fa2324
Move requireApiVersion tests to new format
alcaeus Sep 22, 2023
81614c0
Move loadbalanced tests to new format
alcaeus Sep 22, 2023
214b8f5
Move OCSP tests to new format
alcaeus Sep 22, 2023
7443ace
Generate test and build tasks from templates
alcaeus Sep 25, 2023
eaee3d8
Remove obsolete files
alcaeus Sep 26, 2023
6607c0f
Refactor generation script
alcaeus Sep 26, 2023
37c944c
Remove version identifier from libmongoc task names
alcaeus Sep 26, 2023
5b8d711
Add variant tags to facilitate patch filters
alcaeus Sep 26, 2023
1a92176
Define pull request aliases in evergreen config
alcaeus Sep 26, 2023
d52d13d
Add architecture documentation
alcaeus Sep 26, 2023
1033d86
Use regular expression for PR aliases
alcaeus Sep 27, 2023
0edd44d
Mark generated files as such
alcaeus Sep 27, 2023
7a87c85
Remove unnecessary functions
alcaeus Oct 4, 2023
d19f8ae
Update CI documentation
alcaeus Oct 6, 2023
305e50d
Remove unused YAML tags
alcaeus Oct 9, 2023
86d8369
Document references of libmongoc build tasks
alcaeus Oct 9, 2023
802b0be
Document tags used in tasks
alcaeus Oct 9, 2023
10632e3
Make versions in config generator more readable
alcaeus Oct 9, 2023
2d349d3
Split templates and generated configs
alcaeus Oct 9, 2023
acf1eb7
Add blank lines to task files for readability
alcaeus Oct 9, 2023
d529aca
Add note about libmongoc build tasks in contributing guide
alcaeus Oct 9, 2023
308bfc5
Use real server version in architecture docs
alcaeus Oct 11, 2023
a5efc27
Treat loadbalanced as its own topology in tests
alcaeus Oct 11, 2023
8c0fff7
Kill tasks after 10 minutes
alcaeus Oct 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .evergreen/architecture.md
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Evergreen Build System Architecture

To avoid unnecessary tasks from being started, this project uses two separate build steps. The first step compiles the
PHP extension on the requested operating system and PHP version, caching the result. The second step runs tests on
various platforms, re-using the cached build artifacts from the first step.

## Build Step 1: Compile PHP Extension

The extension is built for all supported PHP versions on all supported platforms. This build step is run for every
commit pushed to the development branch, as well as for stable branches and pull requests to these branches. The build
step includes a smoke test that attempts to load the compiled extension to catch errors that occur during extension
initialisation. If a build step fails, test tasks are skipped for that platform and PHP version.

### Build tasks

Build tasks are generated automatically and included in the main config file. The `build-variants.yml` file contains
the list of supported platforms that the extension is built for. `build-task-groups.yml` defines the task groups that
contain the build tasks. Note there is a separate task that skips PHP 7.4 and 8.0, as these versions do not support
OpenSSL 3 (currently used on RHEL 9+ and Ubuntu 22.04+).

## Build Step 2: Run Tests

Tests are not offered for all platforms, and only select tasks are run for commits and pull requests. First, all
combinations of MongoDB topologies (standalone, replica set, sharded cluster, and load balanced) are run for all
supported MongoDB versions. Then, all PHP versions are tested against the latest stable version of MongoDB. Finally,
a last build variant runs tests on PHP 8.2 against the latest stable version of MongoDB using different libmongoc
versions.

## Modifying generated tasks

Most build and test tasks are generated using the `generate-config.php` script and referenced using tags. To modify the
configuration, change the appropriate `_template-<build>.yml` file in the `build` or `test` directory, then run the
generator script to update the files. The generator script will output a list of include statements that can be copied
to the main `config.yml` file to include all generated tasks.

## Adding new tasks

Adding new tasks for a new MongoDB version or PHP version is done by modifying `generate-config.php` and including the
version in the corresponding arrays, then regenerating configuration. Note that obsolete files will not be deleted
automatically, but they won't be in the list of include files printed by the generator script.

## Patch Aliases

Aliases are configured in the main Evergreen configuration file. `github_pr_aliases` defines aliases for tasks to be run
for every pull request.

## Task Tags

Tasks are tagged to allow for a better selection in build variants. The following tags are used for build tasks:
- `build`: All build tasks are tagged with `build`.
- `build-libmongoc`: These build tasks build with different libmongoc versions.
- `php<version>`: These tags allow selection based on PHP version, e.g. `php8.2`.

Test tasks use the following tags:
- `local`: All tasks that run a local MongoDB cluster for testing.
- `<version>`: These tags allow selection based on MongoDB version, e.g. `7.0`.
- `standalone`, `replicaset`, `sharded`: These tags allow selection based on the MongoDB topology.
- `loadbalanced`: Allows for selecting tests using a load balancer
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that templates/test/load-balanced.yml mixes this tag with "sharded". Is that necessary since LB should already imply a sharded topology? Would it make sense to consider this a topology and fold it into the previous line?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the sharded tag from load balanced tests, effectively treating loadbalanced like its own topology.

- `ocsp`: Used for all OCSP tasks
- `versioned_api`: Used for tests that use the stable API
- `skip_crypt_shared`: These tasks skip installing the shared library to test with libmongocrypt
- `nodb`: These tasks do not rely on a local database
- `atlas`: These tasks work on a MongoDB Atlas cluster
- `storage-engines`: Tag used for tasks that test the `inmemory` and `mmapv1` storage engines
24 changes: 21 additions & 3 deletions .evergreen/compile-unix.sh
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
#!/bin/sh
#!/bin/bash
set -o errexit # Exit the script with error if any of the commands fail

# Find PHP binary path for the requested version
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was moved over so we can run compile for multiple PHP versions on the same box if we wanted to.

jmikola reacted with thumbs up emoji
if [ -z "$PHP_PATH" ]; then
if [ -d "/opt/php/${PHP_VERSION}-64bit/bin" ]; then
PHP_PATH="/opt/php/${PHP_VERSION}-64bit/bin"
else
# Try to find the newest version matching our constant
PHP_PATH=`find /opt/php/ -maxdepth 1 -type d -name "${PHP_VERSION}*-64bit" -print | sort -V -r | head -n 1`/bin
fi
fi

if [ ! -x "$PHP_PATH/php" ]; then
echo "Could not find PHP binaries for version ${PHP_VERSION}. Listing available versions..."
ls -1 /opt/php
exit 1
fi

PATH="$PHP_PATH:$PATH"

# Supported/used environment variables:
# MARCH Machine Architecture. Defaults to lowercase uname -m
# LIBMONGOC_VERSION Optional libmongoc version (regenerate version file if set)
Expand Down Expand Up @@ -45,7 +63,7 @@ case "$OS" in
esac

# Report the current PHP version
echo "PHP: `php --version | head -1`"
echo "PHP: `php --version | head -n 1`"

phpize
./configure --enable-mongodb-developer-flags
Expand All @@ -55,4 +73,4 @@ if [ -n "$LIBMONGOC_VERSION" ]; then
make libmongoc-version-current
fi

make
make test TESTS="tests/smoketest.phpt"
Loading

AltStyle によって変換されたページ (->オリジナル) /