-
Notifications
You must be signed in to change notification settings - Fork 66
Releases: postgres-ai/database-lab-engine
Database Lab Engine v3.1.0
Database Lab Engine v3.1.0
DLE 3.1 has many improvements and several new features, such as native support of pgBackRest and the ability to configure the timezone when using CLI.
In this release, the development team has focused on the Database Lab Engine community, making it easier to get help or contribute. The team greets all new contributors: @Nikolay Devxx, @asotolongo, @Tanya301, @denis-boost, @pietervincken, @ane4ka
Community news
- 🌠 DLE repository on GitHub now has 1,100+ stars; many thanks to everyone who supports the project in any way
- 💥 Pieter Vincken has published a blog post describing their experience of using DLE: "Testing with production data made easy"
- 📈 The Twitter account has reached 400 followers: @Database_Lab
- 🎉 DLE now has 15 contributors. More contributions are welcome! See "good first issues"
- 🥇 Please consider various ways to contribute – read CONTRIBUTING.md
The README.md has been significantly improved. It now has four translations, with more coming in the future. Many thanks to our README translation contributors! It helps engineers worldwide understand DLE concepts and improve the development experience by building powerful dev&test environments.
- Ukrainian / Українська (🙏 @denis-boost)
- Spanish / Español (🙏 @asotolongo)
- Russian / Pусский (🙏 @Tanya301)
- German / Deutsch (🙏 @ane4ka)
Migration notes
postgresai/dblab-server:3.1.0) uses ZFS 2.1.2. If you need to upgrade DLE from a previous version that worked with a ZFS pool 0.8.x (default for previous DLE versions), consider the following options:
- Option 1: upgrade your system to use ZFS 2.1, optionally upgrade your pool (
zpool upgrade dblab_pool), and then upgrade DLE to use the default imagepostgresai/dblab-server:3.1.0 - Option 2: postpone the ZFS upgrade, stay on ZFS 0.8, and upgrade DLE to version 3.1 using a special image,
postgresai/dblab-server:3.1.0-zfs0.8
Changelog
New features
- Native support for pgBackRest as a tool to restore data from archives (physical mode, including continuously updated state) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/442, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/454 (:bow: contribution by @Nikolay Devxx)
- Allow configuring timezone in DLE CLI configuration to improve the experience of using DLE in CI/CD pipelines https://gitlab.com/postgres-ai/database-lab/-/merge_requests/486, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/502
- Automatically clean up failed clones with their sessions (for example, errors during the Postgres clone configuration – in the past, in such cases, DLE lost control over such clones and they could not be removed, even using an API request) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/405, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/404
- For logical mode, during dump/restore, DLE now cleans up only those files or directories that are specified to be restored https://gitlab.com/postgres-ai/database-lab/-/merge_requests/410
- Improved support of multiple DLEs running on the same machine: Take the pool name into account when determining the clone existence https://gitlab.com/postgres-ai/database-lab/-/merge_requests/471
- Rework README, add translations https://gitlab.com/postgres-ai/database-lab/-/merge_requests/411, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/423, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/467, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/478, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/481, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/497
- CONTRIBUTING.md: Describe components and Makefile targets https://gitlab.com/postgres-ai/database-lab/-/merge_requests/413
Improvements and fixes
- Reset the snapshot state atomically to avoid the moments in time when the DLE does not have a snapshot https://gitlab.com/postgres-ai/database-lab/-/merge_requests/406
- Skip connection if the DLE container is already connected to an internal network. This solves the problem when in some cases, DLE cannot start because the DLE container is already connected to the bridge network https://gitlab.com/postgres-ai/database-lab/-/merge_requests/416
- Optimize snapshot list. This has to improve the performance of
/snapshotAPI calls which could take many seconds in the past https://gitlab.com/postgres-ai/database-lab/-/merge_requests/426 - ZFS version 2.1.2 inside DLE container (optional, 0.8 remains the default in DLE 3.x) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/343, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/477
- Do not specify the logerrors extension by default https://gitlab.com/postgres-ai/database-lab/-/merge_requests/485
- Replace the "standby_mode" parameter with "recovery_target" in the DLE configuration for the promotion stage https://gitlab.com/postgres-ai/database-lab/-/merge_requests/484
- Use the correct path to the data directory for the promote instance https://gitlab.com/postgres-ai/database-lab/-/merge_requests/503
- Fix wrong server version in the DLE response https://gitlab.com/postgres-ai/database-lab/-/merge_requests/508
- Avoid losing the snapshot list on configuration reload https://gitlab.com/postgres-ai/database-lab/-/merge_requests/506
- In logical mode, dump only databases to which user have access https://gitlab.com/postgres-ai/database-lab/-/merge_requests/501
- UI: Add clone port to clones list https://gitlab.com/postgres-ai/database-lab/-/merge_requests/447
- UI: Correct status detection https://gitlab.com/postgres-ai/database-lab/-/merge_requests/449
- UI: Add the total size of clones to instance page header https://gitlab.com/postgres-ai/database-lab/-/merge_requests/450
- UI: Fetch failed on connection refused calls https://gitlab.com/postgres-ai/database-lab/-/merge_requests/459
- UI: Adjust DLE stats labels https://gitlab.com/postgres-ai/database-lab/-/merge_requests/461
- UI (Platform only): Fix command page scroll https://gitlab.com/postgres-ai/database-lab/-/commit/992b0ecc4f43a578eeadd3818d8824e470b78009
- UI (Platform only): Fix markdown parser https://gitlab.com/postgres-ai/database-lab/-/merge_requests/496
Internal
- Set up GPG to have signed commits in GitPod https://gitlab.com/postgres-ai/database-lab/-/merge_requests/407
- Use GitHub CodeQL Analysis https://gitlab.com/postgres-ai/database-lab/-/merge_requests/409
- Define rules to reduce the running time of integration tests - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/414
- Upgrade the default Docker image - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/412, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/476
- Add .whitesource configuration file - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/420
- Rework LICENSE files - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/429
- Don't count .js in repo stats - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/430
- Rework CI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/425, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/431, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/453, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/457, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/493, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/495, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/499
- Describe how
dataStateAtis determined - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/356 - Isolate Database Lab Engine sources - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/443
- Update dependencies for security reasons - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/439, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/445, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/456, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/458, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/466, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/451, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/452, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/469, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/468, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/472, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/504
- Make UI source code publicly available https://gitlab.com/postgres-ai/database-lab/-/merge_requests/421
- Update Go to 1.18 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/498
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and...
Assets 2
Database Lab Engine v3.0.3
a59e584 DLE 3.0.3 is a backward-compatibility update to ensure that the DLE image is compatible with existing datasets (ZFS 0.8.*). In DLE 3.0.2, in the Docker image ZFS version was prematurely switched to 2.*. The release 3.0.3 mitigates it. There are plans to switch to ZFS 2.* in the future major versions of DLE.
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository on GitLab or GitHub:
Changelog
Improvements and fixes
- Explicitly set ZFS version in DLE – ZFS 0.8.4: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/492
Internal
- Update the DLE version in the SwaggerUI spec: https://gitlab.com/postgres-ai/database-lab/-/commit/a59e5845dca510a2a3b929ba1834ce038e5a6936
Migration notes
Make sure the DLE running command contains the environment variable --env DOCKER_API_VERSION="1.41".
The maximum supported Docker API version is 1.41, but the client version (1.42) is too new.
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions v3.0.2 and v3.0.3: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.2...v3.0.3
Assets 2
Database Lab Engine v3.0.2
dbd8d1f DLE 3.0.2 is a security update, mitigating multiple CVEs reported by various security inspection tools. There are no known critical vulnerabilities that are fixed or are still unfixed in this release but upgrading at the earliest convenience is recommended.
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository on GitLab or GitHub:
Changelog
Internal
- Fix old and indirect dependencies to avoid scanner warnings - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/466
- Upgrade engine libs to mitigate multiple CVEs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/458
- Mitigate CVE scanner warnings related to SwaggerUI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/472
- Update base Docker images to 20.10 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/476
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions v3.0.1 and v3.0.2: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.1...v3.0.2
Assets 2
Database Lab Engine v3.0.1
9422003 The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository: https://gitlab.com/postgres-ai/database-lab.
Changelog
Improvements and fixes
- Clean up only files or directories that are specified to be restored - https://gitlab.com/postgres-ai/database-lab/-/issues/313
- Remove clone with the FATAL status during idleness check - https://gitlab.com/postgres-ai/database-lab/-/issues/316
- Reload snapshots state atomically - https://gitlab.com/postgres-ai/database-lab/-/issues/317
- Do not connect the DLE container to an internal network if it is already connected - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/416
- Rework README, add CODE_OF_CONDUCT.md, CONTRIBUTING.md, SECURITY.md - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/411, https://gitlab.com/postgres-ai/database-lab/-/issues/319
Internal
- Set up GPG to have signed commits in Gitpod - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/407
- Try GitHub CodeQL Analysis - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/409
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions v3.0.0 and v3.0.1: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.0...v3.0.1
Assets 2
Database Lab Engine v3.0.0
Database Lab Engine (DLE) is an open-source technology that enables thin cloning for PostgreSQL. Thin clones are exceptionally useful when you need to scale the development process. DLE can manage dozens of independent clones of your database on a single machine, so each engineer or automation process works with their very own database provisioned in seconds without extra costs.
DLE v3.0.0 includes more than 100 individual changes. This is the most advanced open-source software ever released that empowers development, testing, and troubleshooting environments for fast-growing projects. The use of Database Lab Engine 3.0 provides a competitive advantage to companies via implementing the "Shift-left testing" approach in software development.
Read the blog post about this release: https://postgres.ai/blog/20211221-dle-3-0-0-brings-ui-and-persistent-clones
Among major changes:
- UI included to the core, it allows working with a single DLE instance,
- persistent clones: clones now survive DLE (or VM) restart,
- for the "logical" data provisioning mode: the ability to switch reset clone's state using a snapshot from different pool/dataset,
- better logging, easier configuration,
- improvements for the cases with multiple DLEs are running a single machine,
- PostgreSQL 14 support.
Please note that upgrading from an older version requires some actions from the DLE administrator (see "Migration notes" below).
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository: https://gitlab.com/postgres-ai/database-lab.
Changelog
New features
- Do not lose clones when DLE is restarted - https://gitlab.com/postgres-ai/database-lab/-/issues/188
- Keep clone containers after VM reboot - https://gitlab.com/postgres-ai/database-lab/-/issues/310
- Run a new satellite-container serving the Embedded UI app - https://gitlab.com/postgres-ai/database-lab/-/issues/302
- Provide information in API about available pools - https://gitlab.com/postgres-ai/database-lab/-/issues/275
- Support PostgreSQL 14 - https://gitlab.com/postgres-ai/database-lab/-/issues/290
- Enable DLE Telemetry (optional) - https://gitlab.com/postgres-ai/database-lab/-/issues/299, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/373
- Provide the DLE container name to control container labels - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/365
- Add the ability to select a working pool explicitly - https://gitlab.com/postgres-ai/database-lab/-/issues/306
- Configure IP address to listen HTTP connections for Embedded UI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/378
- Immediate start HTTP server in DLE and provide the actual instance status - https://gitlab.com/postgres-ai/database-lab/-/issues/307
- Start DLE and Embedded UI even if no data pool available - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/393
- Allow resetting clone to a snapshot from a different pool - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/345
Improvements and fixes
- Allow an empty body in the
resetcommand - https://gitlab.com/postgres-ai/database-lab/-/issues/291 - Do not mark the first pool active by default - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/346
- Revise port range availability after deleting invalid clone containers - https://gitlab.com/postgres-ai/database-lab/-/issues/296
- Clean up the
dumpLocationdirectory before taking a dump - https://gitlab.com/postgres-ai/database-lab/-/issues/294 - Select the latest snapshot according to the data timestamp - https://gitlab.com/postgres-ai/database-lab/-/issues/304
- Disable full refresh by default in "physical" mode - https://gitlab.com/postgres-ai/database-lab/-/issues/303
- Consider the preliminary snapshot when calculating the size of the snapshot - https://gitlab.com/postgres-ai/database-lab/-/issues/305
- Adjust the payload parameter in telemetry requests - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/379
- Remove the host parameter from the server configuration since it is always runs inside a Docker container - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/388
- Create pg_stat_kcache extension before running of an observing session - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/395
- Recognize errors from external tools and mention them in logs - https://gitlab.com/postgres-ai/database-lab/-/issues/293
- Move the instance_id file to the metadata directory to run multiple DLEs on the same machine - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/396
- Be resistant to an absent snapshot: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/399
- Refresh clone metadata in the instance status - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/400
- Manage full data refresh overlapping - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/402
- Clean up components of a failed clone session - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/404
Internal
- Use semantic Git tags to specify code revision and build Docker images - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/341
- Install a specific version of Envoy as a workaround of using a Postgres Proxy filter - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/349
- Refactor Instance status DLE API - https://gitlab.com/postgres-ai/database-lab/-/issues/297
- Update the Swagger API schema - https://gitlab.com/postgres-ai/database-lab/-/issues/297, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/364
- Extend testing the DLE feature-revision for all supported PostgreSQL versions - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/350, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/375
- Make configuration management more reliable in the TF-template - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/366
- Refactor the project packages structure - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/367
- Upgrade the project dependencies - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/369
- Upgrade the Swagger UI version to 4.1.0 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/370
- Upgrade Go version to 1.17 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/371
- Update golangci-lint to version 1.42 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/374
- Check suppressed linter warnings - https://gitlab.com/postgres-ai/database-lab/-/issues/38
- Make synthetic tests configuration more reliable - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/372
- Exclude side effects while running integration tests in dev and staging environments - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/375
- Upgrade the DLE module version to v3 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/377
- Consider the leading letter
vwhile comparing Git tags to run CI jobs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/382 - Check the state of containers in tests after waiting loops and fail immediately, if there is still no success - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/390
- Build and publish actual CLI binaries on merging MR - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/391
Migration notes
The Database Lab configuration contains a new section, embeddedUI, to run UI application that allows observing the current state of the instance and performing basic clone operations:
embeddedUI: enabled: true dockerImage: "postgresai/ce-ui:latest" host: "" port: 2346
Note, this application is delivered as a separate Docker container ("Embedded UI") and is controlled by DLE.
Starting with version v3.0.0, DLE collects non-personally identifiable telemetry data.
The Engine uses anonymous usage data to assist users and improve product quality.
it is possible to manage the configuration in the section global:
global: telemetry: enabled: true url: "https://postgres.ai/api/general" ...
Please leave it enabled if possible – this will contribute to Database Lab development.
See more details on the Telemetry page in the docs
Now it is available to select a specific pool to work with, use a new option selectedPool in the poolManager section
poolManager: selectedPool: "dblab_pool" ...
Starting with version v3.0.0, Database Lab Engine can be running with an empty verification token, which is not recommended by default.
In such a case DLE warns about empty token in instance logs and API will not require any credentials. Be sure that this option is configured correctly
In order to make the management of service containers easier, special labels have been added to them:
- `dblab_instance_id`: <dle_instance_id>
- `dblab_engine_name`: <dle_container_name>
For example, there is a snippet of the docker inspection response of a sync instance:
"Labels": { "dblab_control": "dblab_sync", "dblab_instance_id": "c64mvrvku3086hlp2e6g", "dblab_engine_name": "dblab_server", "maintainer": "postgres.ai" }
See more examples of configuration files in the Database Lab repository
The Docker image of DLE v3.0.0: postgresai/dblab-server:3.0.0
Launch command example:
sudo docker run \ --name dblab_server \ --label dblab_control \ --privileged \ --publish 127.0.0.1:2345:2345 \ --volume /var/run/docker.sock:/var/run/docker.sock \ --volume /var/l...
Assets 2
DLE v3.0.0 Release Candidate 2
The second release candidate of DLE version 3.0.0, a major release with many changes and improvements. See the release notes for RC1: https://gitlab.com/postgres-ai/database-lab/-/tags/v3.0.0-rc.1
Compared to RC1, this version fixes multiple bugs, points on logical inconsistency, and issues based on users' feedback during RC1 testing:
- Keep clone containers after VM reboot (previously, clone persistency worked only for container restarts) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/389
- Postgres 14 as default everywhere https://gitlab.com/postgres-ai/database-lab/-/merge_requests/384
- Remove the API's
hostparameter from the server configuration since it is always running inside a Docker container https://gitlab.com/postgres-ai/database-lab/-/merge_requests/388 - Describe the meaning of empty
verificationTokenin configs https://gitlab.com/postgres-ai/database-lab/-/merge_requests/388 - Start UI as soon as possible, even if the other processes are not finished. Provide actual instance status in API https://gitlab.com/postgres-ai/database-lab/-/merge_requests/383
- Start UI even if no data pool is available https://gitlab.com/postgres-ai/database-lab/-/merge_requests/393
- Create pg_stat_kcache extension before observability activities for DB Migration Checker start https://gitlab.com/postgres-ai/database-lab/-/merge_requests/395
- Recognize errors from external tools and expose them in logs (example: pg_dump attempt failed with some error) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/386
- Move the
instance_idfile to the metadata directory to avoid conflicts when multiple DLEs are running on the same machine https://gitlab.com/postgres-ai/database-lab/-/merge_requests/396 - Show correct DLE start time in API responses https://gitlab.com/postgres-ai/database-lab/-/merge_requests/397
- Internal changes (CI, workflow improvements, internal naming)
- match tags with leading
vhttps://gitlab.com/postgres-ai/database-lab/-/merge_requests/382 - change UI docker tag in sample configs https://gitlab.com/postgres-ai/database-lab/-/merge_requests/385
- improve the way how Postgres availability is checked in automated tests https://gitlab.com/postgres-ai/database-lab/-/merge_requests/390
- various improvements for CI/CD pipelines https://gitlab.com/postgres-ai/database-lab/-/merge_requests/391
- rename Local UI to Embedded UI and ce-ui Docker image
- match tags with leading
- Multiple fixes and improvements in the Terraform module template code to work with DLE 3.0.0 https://gitlab.com/postgres-ai/terraform-postgres-ai-database-lab/-/compare/v0.1.1...master
Migration notes
Upgrading from version 2.5.x and earlier require some actions from the DLE administrator; see the migration notes for v3.0.0-rc.1: https://gitlab.com/postgres-ai/database-lab/-/tags/v3.0.0-rc.1#migration-notes
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions v3.0.0-rc.1 and v3.0.0-rc.2: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.0-rc.1...v3.0.0-rc.2
Assets 2
DLE v3.0.0 Release Candidate 1
DLE v3.0.0-rc.1 includes several major changes and many improvements. Most notable changes are:
- persistent clones: clones now survive DLE (or VM) restart,
- GUI included to the core, it allows working with a single DLE instance,
- PostgreSQL 14 support.
Please note that upgrading from an older version requires some actions from the DLE administrator (see "Migration notes" below).
The list below has "WIP" status. The final list for the 3.0 major version release may include some changes.
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository: https://gitlab.com/postgres-ai/database-lab.
Changelog
New features
- Do not lose clones when DLE is restarted - https://gitlab.com/postgres-ai/database-lab/-/issues/188
- Run a new satellite-container serving the Local UI app - https://gitlab.com/postgres-ai/database-lab/-/issues/302
- Provide information in API about available pools - https://gitlab.com/postgres-ai/database-lab/-/issues/275
- Support PostgreSQL 14 - https://gitlab.com/postgres-ai/database-lab/-/issues/290
- Enable DLE Telemetry (optional) - https://gitlab.com/postgres-ai/database-lab/-/issues/299, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/373
- Provide the DLE container name to control container labels - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/365
- Add the ability to select a working pool explicitly - https://gitlab.com/postgres-ai/database-lab/-/issues/306
- Configure IP address to listen HTTP connections for Local UI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/378
Improvements and fixes
- Allow an empty body in the
resetcommand - https://gitlab.com/postgres-ai/database-lab/-/issues/291 - Do not mark the first pool active by default - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/346
- Revise port range availability after deleting invalid clone containers - https://gitlab.com/postgres-ai/database-lab/-/issues/296
- Allow resetting clone to a snapshot from a different pool - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/345
- Clean up the
dumpLocationdirectory before taking a dump - https://gitlab.com/postgres-ai/database-lab/-/issues/294 - Select the latest snapshot according to the data timestamp - https://gitlab.com/postgres-ai/database-lab/-/issues/304
- Disable full refresh by default in "physical" mode - https://gitlab.com/postgres-ai/database-lab/-/issues/303
- Consider the preliminary snapshot when calculating the size of the snapshot - https://gitlab.com/postgres-ai/database-lab/-/issues/305
- Adjust the payload parameter in telemetry requests - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/379
Internal
- Use semantic Git tags to specify code revision and build Docker images - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/341
- Install a specific version of Envoy as a workaround of using a Postgres Proxy filter - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/349
- Refactor Instance status DLE API - https://gitlab.com/postgres-ai/database-lab/-/issues/297
- Update the Swagger API schema - https://gitlab.com/postgres-ai/database-lab/-/issues/297, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/364
- Extend testing the DLE feature-revision for all supported PostgreSQL versions - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/350, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/375
- Make configuration management more reliable in the TF-template - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/366
- Refactor the project packages structure - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/367
- Upgrade the project dependencies - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/369
- Upgrade the Swagger UI version to 4.1.0 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/370
- Upgrade Go version to 1.17 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/371
- Update golangci-lint to version 1.42 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/374
- Check suppressed linter warnings - https://gitlab.com/postgres-ai/database-lab/-/issues/38
- Make synthetic tests configuration more reliable - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/372
- Exclude side effects while running integration tests in dev and staging environments - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/375
- Upgrade the DLE module version to v3 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/377
Migration notes
The Database Lab configuration contains a new section localUI to run UI application that allows observing the current state of the instance and performing basic clone operations:
localUI: enabled: true dockerImage: "postgresai/basic-ui:latest" host: "" port: 2346
Note, this application is delivered as a separate Docker container (Local UI) and running by the Database Lab Engine.
Since v3.0.0-rc.1, DLE collects non-personally identifiable telemetry data.
The Engine uses anonymous usage data to assist users and improve product quality.
it is possible to manage the configuration in the section global:
global: telemetry: enabled: true url: "https://postgres.ai/api/general" ...
Please leave it enabled if possible – this will contribute to Database Lab development.
See more details on the Telemetry page in the docs
Now it is available to select a specific pool to work with, use a new option selectedPool in the poolManager section
poolManager: selectedPool: "dblab_pool" ...
Since version v3.0.0-rc.1, Database Lab Engine can be running with an empty verification token, which is not recommended by default.
In such a case DLE warns about empty token in instance logs and API will not require any credentials. Be sure that this option is configured correctly
In order to make the management of service containers easier, special labels have been added to them:
- `dblab_instance_id`: <dle_instance_id>
- `dblab_engine_name`: <dle_container_name>
For example, there is a snippet of the docker inspection response of a sync instance:
"Labels": { "dblab_control": "dblab_sync", "dblab_instance_id": "c64mvrvku3086hlp2e6g", "dblab_engine_name": "dblab_server", "maintainer": "postgres.ai" }
See more examples of configuration files in the Database Lab repository
The Docker image of DLE 3.0.0-rc.1: registry.gitlab.com/postgres-ai/database-lab/dblab-server:test-3-0-0-rc-1
Launch command example:
sudo docker run \ --name dblab_server \ --label dblab_control \ --privileged \ --publish 127.0.0.1:2345:2345 \ --volume /var/run/docker.sock:/var/run/docker.sock \ --volume /var/lib/dblab:/var/lib/dblab/:rshared \ --volume ~/.dblab/engine/configs:/home/dblab/configs:ro \ --volume ~/.dblab/engine/meta:/home/dblab/meta \ --env DOCKER_API_VERSION=1.39 \ --detach \ --restart on-failure \ registry.gitlab.com/postgres-ai/database-lab/dblab-server:test-3-0-0-rc-1
If you have problems or questions, please contact our communities for help: https://postgres.ai/docs/questions-and-answers#where-to-get-help
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions 2.5.0 and v3.0.0-rc.1: https://gitlab.com/postgres-ai/database-lab/-/compare/2.5.0...v3.0.0-rc.1