Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

setup, config, selenium_driverless, images, comments #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
milahu wants to merge 28 commits into timf34:main
base: main
Choose a base branch
Loading
from milahu:main
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
c520fb3
add setup.py
milahu Dec 27, 2025
c73855e
add parameters: config email password
milahu Dec 27, 2025
15fc25c
add parameters: assets-dir author-template
milahu Dec 28, 2025
8d7676f
use selenium_driverless
milahu Dec 28, 2025
4af8b45
replace existing files
milahu Dec 28, 2025
4b8598a
fixup: assets-dir
milahu Dec 28, 2025
5811bb5
download images
milahu Dec 28, 2025
153746f
download comments
milahu Dec 28, 2025
591fa86
handle removed comments
milahu Dec 28, 2025
1458d78
add debug comment
milahu Dec 28, 2025
07e4c1d
write JSON files
milahu Dec 28, 2025
f17ad50
use output filepath format strings
milahu Dec 29, 2025
38875f8
add json_dump_kwargs
milahu Dec 29, 2025
2ef534c
fix class SubstackScraper
milahu Dec 30, 2025
ad84f46
add parameter offline
milahu Dec 30, 2025
94192ca
fix paths in posts.json
milahu Dec 30, 2025
6cccf75
add repost_count to posts_data
milahu Dec 30, 2025
d311b56
add post_json to posts_data
milahu Dec 30, 2025
d7ff582
add generate_main_md_file
milahu Dec 30, 2025
cbf9cbe
fix post links: remove ../ prefix
milahu Dec 30, 2025
8d24a80
fix post links: remove target="_blank"
milahu Dec 30, 2025
c138b87
store ISO format date in posts.json
milahu Dec 30, 2025
2acf823
fix regex pattern in process_markdown_images
milahu Dec 30, 2025
e5b0bdd
download images: add resolve_image_url
milahu Dec 30, 2025
0997609
fix type of like_count
milahu Dec 30, 2025
4ecb5e2
fix url loop in scrape_posts
milahu Dec 31, 2025
e5b0f17
remove parameter offline
milahu Dec 31, 2025
55b7e91
add process_markdown_links
milahu Dec 31, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions config.py
View file Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions data/README.md
View file Open in desktop

This file was deleted.

3 changes: 1 addition & 2 deletions requirements.txt
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
bs4==0.0.1
html2text==2020年1月16日
requests==2.31.0
selenium==4.16.0
selenium-driverless
tqdm==4.66.1
webdriver_manager==4.0.1
Markdown==3.6
View file Open in desktop
Empty file.
File renamed without changes.
File renamed without changes.
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ function populateEssays(data) {
const essaysContainer = document.getElementById('essays-container');
const list = data.map(essay => `
<li>
<a href="../${showHTML ? essay.html_link : essay.file_link}" target="_blank">${essay.title}</a>
<a href="${showHTML ? essay.html_link : essay.file_link}">${essay.title}</a>
<div class="subtitle">${essay.subtitle}</div>
<div class="metadata">${essay.like_count} Likes - ${essay.date}</div>
</li>
Expand Down
Loading

AltStyle によって変換されたページ (->オリジナル) /