Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit c66c7fc

Browse files
Merge pull request #1 from NarrativeScience/init-code-commit
Initial code commit
2 parents 8e07c0e + 2051371 commit c66c7fc

File tree

12 files changed

+1013
-42
lines changed

12 files changed

+1013
-42
lines changed

‎.circleci/config.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ commands:
1616
jobs:
1717
test:
1818
docker:
19-
- image: circleci/python:3.7
19+
- image: circleci/python:3.6
2020
steps:
2121
- ghpr/build-prospective-branch
2222
- install-deps
@@ -34,7 +34,7 @@ jobs:
3434
when: on_fail
3535
publish:
3636
docker:
37-
- image: circleci/python:3.7
37+
- image: circleci/python:3.6
3838
steps:
3939
- checkout
4040
- run: pip install flit

‎README.md‎

Lines changed: 42 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,67 @@
1-
# my-package-name
1+
# async-task-qeue
22

3-
[![](https://img.shields.io/pypi/v/my-package-name.svg)](https://pypi.org/pypi/my-package-name/) [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
3+
[![](https://img.shields.io/pypi/v/sfn-workflow-client.svg)](https://pypi.org/pypi/async-task-queue/) [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
44

5-
<!-- Short description of the package -->
5+
In-memory FIFO queue for concurrent task execution. Used to execute tasks concurrently with optional control (via semaphore) over the max number of tasks running at the same time.
66

77
Features:
88

9-
- <!-- list of features -->
9+
- Queue processing summary logging
10+
- Introspection of failed, retried, and succeeded tasks
11+
- Task retries (optional)
12+
- Task execution timeout (optional)
13+
- Queue processing with semaphore (optional)
14+
- Batch size control (optional)
1015

11-
Table of Contents:
16+
TOC:
1217

1318
- [Installation](#installation)
1419
- [Guide](#guide)
1520
- [Development](#development)
1621

1722
## Installation
1823

19-
my-package-name requires Python 3.6 or above.
24+
async-task-queue requires Python 3.6 or above.
2025

2126
```bash
22-
pip install my-package-name
27+
pip install async-task-queue
2328
```
2429

2530
## Guide
2631

27-
<!-- Subsections explaining how to use the package -->
32+
```python
33+
import logging
34+
from async_task_queue import AsyncTask, AsyncTaskQueue
35+
36+
# Initialize a logger
37+
logger = logging.getLogger("foo")
38+
39+
# Initialize an AsyncTaskQueue where:
40+
# - At most 5 tasks are running concurrently
41+
# - Number of tasks executing concurrently should be limited by a semaphore
42+
# - Failed tasks should be retried (default behavior)
43+
# - Executing the tasks queued should timeout and be cancelled after 5 minutes
44+
task_queue = AsyncTaskQueue(
45+
logger,
46+
use_semaphore=True,
47+
batch_size=5,
48+
execution_timeout=300
49+
)
50+
51+
# Add async tasks to the queue
52+
task_queue.enqueue(
53+
[
54+
AsyncTask(some_coroutine, arg) for arg in some_args
55+
]
56+
)
57+
58+
# Start processing the queue
59+
await task_queue.execute()
60+
```
2861

2962
## Development
3063

31-
To develop my-package-name, install dependencies and enable the pre-commit hook:
64+
To develop async-task-queue, install dependencies and enable the pre-commit hook:
3265

3366
```bash
3467
pip install pre-commit tox

‎pyproject.toml‎

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,23 +3,20 @@ requires = ["flit_core >=2,<3"]
33
build-backend = "flit_core.buildapi"
44

55
[tool.flit.metadata]
6-
module = "my_package_name"
7-
author = "Jerry Seinfeld"
8-
author-email = "jseinfeld@narrativescience.com"
9-
home-page = "https://github.com/NarrativeScience/my-package-name"
6+
module = "async_task_queue"
7+
author = "Celestine Kao"
8+
author-email = "ckao@narrativescience.com"
9+
home-page = "https://github.com/NarrativeScience/async-task-queue"
1010
license = "BSD-3-Clause"
1111
description-file = "README.md"
12-
requires = [
13-
# Requirements here, e.g. "backoff>=1.8.0,<2",
14-
]
1512
classifiers = [
1613
"Intended Audience :: Developers",
1714
"License :: OSI Approved :: BSD License",
1815
"Programming Language :: Python :: 3",
1916
"Topic :: Software Development :: Libraries :: Python Modules",
2017
]
2118
requires-python = ">=3.6,<4"
22-
keywords = "comma,separated,keywords"
19+
keywords = "asyncio, async queue"
2320

2421
[tool.flit.sdist]
2522
include = ["LICENSE.md"]

‎setup.cfg‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@ force_sort_within_sections=true
1010
include_trailing_comma=true
1111
known_standard_library=typing
1212
known_first_party=
13-
my_package_name
13+
async_task_queue
14+
test_utils
1415
line_length=88
1516
multi_line_output=3
1617
no_lines_before=LOCALFOLDER

‎src/async_task_queue/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
"""In-memory FIFO queue for concurrent task execution"""
2+
from .async_task_queue import AsyncTask, AsyncTaskQueue
3+
4+
__version__ = "0.1.0"

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /