Fetch and filter articles from the web

This repository contains a small python script and library to fetch and filter web articles from different sources. Mostly useful in combination with aggregators, like newsraft.

At the moment only a small selection of mostly german news sources are implemented. By default the site is only parsed by an HTML filter, which is not sufficient for most news sources.

Script

Usage

article_filter.py [-h] [--line-width LINE_WIDTH] [--center | --no-center] [--indentation INDENTATION]
 [--top-padding TOP_PADDING] [--bottom-padding BOTTOM_PADDING] [--pager PAGER]
 url
Fetch, parse and print articles from online magazines
positional arguments:
 url Web url of target article
options:
 -h, --help show this help message and exit
 --line-width LINE_WIDTH
 Max. line width of output
 --center, --no-center
 Center article in terminal window. Overwrites indentation argument.
 --indentation INDENTATION
 Indentation of output
 --top-padding TOP_PADDING
 Number of empty lines printed over article
 --bottom-padding BOTTOM_PADDING
 Number of empty lines printed after article
 --pager PAGER Pager command to use for output (default: 'less')

Library

class WebArticleFilter

Usage

webfilter = WebArticleFilter()
article = webfilter.fetch_and_format(url)
webfilter.print_article(article)

Constructor Arguments

• center: Calculate and overwrite indentation from width of current terminal window (default: False)
• indentation: Indentation of lines printed by print_article method (default: 4)
• top_padding: Empty lines printed before article by print_article method (default: 2)
• bottom_padding: Empty lines printed after article by print_article method (default: 2)
• line_width: Max. line width printed by print_article method (default: 60)
• pager: Select pager command to use for output (default: less)

Methods

fetch_and_format(self, url: str) -> str

Fetches article from url and parses article.

print_article_to_str(self, text: str) -> str

Prints text to string according to rules set in class.