-
-
Notifications
You must be signed in to change notification settings - Fork 161
setup, config, selenium_driverless, images, comments #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
Draft
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
c520fb3
add setup.py
milahu c73855e
add parameters: config email password
milahu 15fc25c
add parameters: assets-dir author-template
milahu 8d7676f
use selenium_driverless
milahu 4af8b45
replace existing files
milahu 4b8598a
fixup: assets-dir
milahu 5811bb5
download images
milahu 153746f
download comments
milahu 591fa86
handle removed comments
milahu 1458d78
add debug comment
milahu 07e4c1d
write JSON files
milahu f17ad50
use output filepath format strings
milahu 38875f8
add json_dump_kwargs
milahu 2ef534c
fix class SubstackScraper
milahu ad84f46
add parameter offline
milahu 94192ca
fix paths in posts.json
milahu 6cccf75
add repost_count to posts_data
milahu d311b56
add post_json to posts_data
milahu d7ff582
add generate_main_md_file
milahu cbf9cbe
fix post links: remove ../ prefix
milahu 8d24a80
fix post links: remove target="_blank"
milahu c138b87
store ISO format date in posts.json
milahu 2acf823
fix regex pattern in process_markdown_images
milahu e5b0bdd
download images: add resolve_image_url
milahu 0997609
fix type of like_count
milahu 4ecb5e2
fix url loop in scrape_posts
milahu e5b0f17
remove parameter offline
milahu 55b7e91
add process_markdown_links
milahu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
2 changes: 0 additions & 2 deletions
config.py
Oops, something went wrong.
3 changes: 0 additions & 3 deletions
data/README.md
Oops, something went wrong.
3 changes: 1 addition & 2 deletions
requirements.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,6 @@ | ||
| bs4==0.0.1 | ||
| html2text==2020年1月16日 | ||
| requests==2.31.0 | ||
| selenium==4.16.0 | ||
| selenium-driverless | ||
| tqdm==4.66.1 | ||
| webdriver_manager==4.0.1 | ||
| Markdown==3.6 |
Empty file added
src/substack2markdown/__init__.py
Empty file.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.