Add option to record forced experiment
Use to_bool on bool env vars
π Fix
- Check for NoneType. [Jose Diaz-Gonzalez]
Other
π Fix
- Be less cute about in-place logic. Closes #279. [Jose Diaz-Gonzalez]
Other
Adding Flask-CORS
CORS for sixpack
π Fix
- π Use correct grep for checking repo status. [Jose Diaz-Gonzalez]
Other
Experiment names must be lowercase
Update participation endpoint argument
add config to enable werkzeug ProxyFix middleware
Add note to make explicit that secret is required, not optional for sixpack-web
Bump Flask-Assets required version to 0.12
Update README.rst
Drop --use-mirrors flag
Refs #248
Catch the expirement error by moving the try/catch around
The design of sixpack was meant to allow easy integration of a/b testing into existing applications, thus we do not have any plans to integrate test whitelisting into the core.
Add pause functionality so we can do actual archiving of the data
Remove unarchive function
Remove debugging code
Adding some test coverage for the archive/pause methods
Covering the all method better
Increasing code coverage for the reset function
Add support for kpi descriptions
Ensure atomic experiment creation.
The transaction wrapping the creation of experiment keys in Experiment.save did not watch any of those keys. Consequently parallel writers could cause duplicate writes against an experiment's mutable values, such as its alternatives list.
This adds a watch guard to the experiment's key. Note that Experiment.is_new_record now checks the existence of the experiment's key rather than its membership in the experiments set; this is so that the code watches a value unique to the experiment and not one shared by all.
On conflict the experiment's traffic fraction is persisted.
fixes to benchmark util's use of sixpack client
Add ability to send metrics to statsd
Currently tracks:
Fix web UI chart loading issue
Handle experiment names with underscore the same way as they are rendered in template. See 'data-experiment' attribute in details.html template
Pin gunicorn and gevent
Tested in production with 17.5 and 19.4, working locally with a range of those versions.
Add gunicorn and gevent
These don't need to be pegged to a specific version, and are confirmed working with gunicorn 17.5 through 19.4.1.
[bug] cast the environment variable to an integer
Various improvements to README.rst
Consistency improvements throughout the README.
Add a configuration value to disable csrf
allow traffic fraction to change in mid-flight
Fix readme heading for 2.0.1
Fix early bailout in existing_alternative for excluded clients
Also added an additional assert to the excluded client test that verifies excluded clients have no existing alternative even after a call to Experiment.get_alternative.
[TRAFFIC] Fix over-recording
Remove round from choose alternative
Rounding the random number generated at choose_alternative is excluding users who happen to get a random number greater or equal to 0.990000.
Do not check traffic fraction for update on every participation.
Redis database is now flushed after every test.
If a participation is requested without a traffic fraction argument, the traffic fraction is no longer assumed to be 1. This caused requests to always fail for experiments with a traffic fraction lower than 1 without explicit argument.
Further, the server no longer defaults the request parameter "traffic_fraction" to 1 but simply leaves it at None. It's up to the model to default this value to 1 only when creating an new experiment.
Catch ValueError during g_stat calculation
There can be cases where the conversions for a given alternative are zero, resulting in a math domain error when taking the log of the value.
Conflicts: sixpack/test/experiment_model_test.py
When traffic_fraction is < 1, some clients get the control alternative. The participations of these excluded clients are not recorded to redis. When there is a conversion request for an excluded client, the conversion is not discarded and recorded to redis. When there are a couple of these conversions by excluded clients, the number of completed conversions becomes bigger than the number of participants, which should never be possible. The computation of the confidence_interval relies on this assumption and fails when the completed_count becomes bigger than participant_count.
The solution is to discard the conversions of excluded clients as well.
Fixing participating typo
Display the number of clients that were excluded from the experiment on the dashboard.
Add sixpack-java to client list