Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit 78af4d5

Browse files
Merge pull request avinashkranjan#2101 from sabhisharma-ise/Fetch_Contributions
added fetch_contributions.py ✅
2 parents d0ee819 + 55ca958 commit 78af4d5

File tree

4 files changed

+212
-1
lines changed

4 files changed

+212
-1
lines changed

‎Fetch_Contributions/README.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Fetch contributions
2+
3+
It's a Python script to fetch the pull requests made by a github user in an Organization. It helps organization members to keep record of a particular
4+
github user requests and also helps the github user to record the pull requests made to the organization.
5+
6+
- Script can be executed as a CLI.
7+
- Takes arguments username and organization.
8+
- Returns the pull requests made by user to the organization into a Markdown file.
9+
- Markdown file consists a table having - "Title of PR" , "Link of PR" , "Status(Merged/Closed/Open)"
10+
11+
## Example Markdown File:
12+
13+
| | Title of PR | Link of PR | Status(Merged/Closed/Open) |
14+
|---:|:----------------------------------------------------------|:----------------------------------------------------------------------|:-----------------------------|
15+
| 0 | Title of the pull request made by user | Link to the Pull request | Status of the pull request |
16+
***
17+
## Setup Instructions:
18+
19+
```bash
20+
git clone <link>
21+
cd Fetch_contributions/
22+
python3 -m venv env
23+
source env/bin/activate
24+
pip install -r requirements.txt
25+
python3 fetch_contributions.py --username <Username> --organization <Organization>
26+
```
Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
import argparse
2+
3+
import pandas as pd
4+
import requests
5+
from bs4 import BeautifulSoup
6+
from lxml import html
7+
from tabulate import tabulate
8+
9+
10+
class Fetch_PullRequests:
11+
"""
12+
Fetches the pull requests of a user in a organization.
13+
"""
14+
def __init__(self, username, organization, filename):
15+
"""
16+
:param username: github user
17+
:param organization: Organisation name
18+
:param filename: filename, it's optional
19+
"""
20+
self.ORG_URL = f"https://github.com/orgs/{organization}/repositories"
21+
self.URL = f"https://github.com/{organization}"
22+
self.organization = organization
23+
self.username = username
24+
self.filename = filename
25+
26+
def _list_of_repositories(self):
27+
"""
28+
Function lists the repositories of the organisation.
29+
30+
Returns
31+
-------
32+
list
33+
lists the repositories
34+
35+
"""
36+
page = requests.get(self.ORG_URL)
37+
tree = html.fromstring(page.content)
38+
number_of_pages = tree.xpath('//*[@id="org-repositories"]/div/div/div[2]/div/em/@data-total-pages')
39+
Repositories = []
40+
if len(number_of_pages) == 0:
41+
Repositories.extend(tree.xpath(
42+
'//*[contains(concat( " ", @class, " " ), concat( " ", "wb-break-all", " " ))]//*[contains(concat( " ", @class, " " ), concat( " ", "d-inline-block", " " ))]/text()'))
43+
else:
44+
for number in range(1, int(number_of_pages[0]) + 1):
45+
page_ = requests.get(self.ORG_URL + f"?page={number}")
46+
tree = html.fromstring(page_.content)
47+
Repositories.extend(tree.xpath(
48+
'//*[contains(concat( " ", @class, " " ), concat( " ", "wb-break-all", " " ))]//*[contains(concat( " ", @class, " " ), concat( " ", "d-inline-block", " " ))]/text()'))
49+
50+
return list(pd.Series(list(set(Repositories))).str.strip().values)
51+
52+
def _extract_pullrequests(self, repo):
53+
"""
54+
Function fetches the pull request of a repo.
55+
56+
Parameters
57+
----------
58+
repo: str
59+
repository name
60+
61+
Returns
62+
-------
63+
pandas dataframe
64+
dataframe consists of columns - "Title to PR", "Link of PR", "Status(Merged/Closed/Open)"
65+
66+
"""
67+
# initializing the lists to store the title, link and status of the pull request
68+
Title = []
69+
Link = []
70+
Status = []
71+
URL = self.URL + f"/{repo}/pulls?q=is%3Apr+author%3A{self.username}"
72+
page = requests.get(URL)
73+
tree = html.fromstring(page.content)
74+
# to determine the number of pages
75+
number_of_pages = tree.xpath('//*[@id="repo-content-pjax-container"]/div/div[6]/div/em/@data-total-pages')
76+
77+
if len(number_of_pages) == 0:
78+
# Title.extend(tree.xpath('//*[contains(concat( " ", @class, " " ), concat( " ", "markdown-title", " " ))]/text()'))
79+
soup = BeautifulSoup(page.text, 'html.parser')
80+
# "Title may contain text in <code> tags. So,to handle it we use beautiful soup.
81+
for tag in soup.find_all('a', attrs={'class': 'markdown-title'}):
82+
Title.append(tag.text.strip())
83+
Link.extend(
84+
tree.xpath('//*[contains(concat( " ", @class, " " ), concat( " ", "markdown-title", " " ))]/@href'))
85+
Status.extend(tree.xpath(
86+
'//*[contains(concat( " ", @class, " " ), concat( " ", "pl-3", " " ))]/span/@aria-label'))
87+
88+
else:
89+
for number in range(1, int(number_of_pages[0]) + 1):
90+
URL = self.URL + f"/{repo}/pulls?page={number}&q=is%3Apr+author%3A{self.username}"
91+
page = requests.get(URL)
92+
tree = html.fromstring(page.content)
93+
94+
# Title.extend(tree.xpath('//*[contains(concat( " ", @class, " " ), concat( " ", "markdown-title", " " ))]/text()'))
95+
soup = BeautifulSoup(page.text, 'html.parser')
96+
# Names = tree.xpath(
97+
# '//*[contains(concat( " ", @class, " " ), concat( " ", "opened-by", " " ))]//*[contains(concat( " ", @class, " " ), concat( " ", "Link--muted", " " ))]/text()')
98+
99+
for tag in soup.find_all('a', attrs={'class': 'markdown-title'}):
100+
Title.append(tag.text.strip())
101+
Link.extend(tree.xpath(
102+
'//*[contains(concat( " ", @class, " " ), concat( " ", "markdown-title", " " ))]/@href'))
103+
Status.extend(tree.xpath(
104+
'//*[contains(concat( " ", @class, " " ), concat( " ", "pl-3", " " ))]/span/@aria-label'))
105+
106+
Data = {
107+
"Title to PR": Title,
108+
"Link of PR": Link,
109+
"Status(Merged/Closed/Open)": Status
110+
}
111+
112+
# creating a dataframe with the above dictionary
113+
dataframe = pd.DataFrame.from_dict(Data)
114+
# dataframe.head()
115+
116+
# make necessary changes to the columns of dataframe before returning it
117+
dataframe['Status(Merged/Closed/Open)'] = dataframe['Status(Merged/Closed/Open)'].astype(str).str.replace(
118+
" pull request",
119+
"", regex=False)
120+
if dataframe['Link of PR'].dtype!='O':
121+
dataframe['Link of PR'] = dataframe['Link of PR'].astype(str)
122+
dataframe['Link of PR'] = 'https://github.com' + dataframe['Link of PR']
123+
124+
return dataframe
125+
126+
def get_pullrequests(self):
127+
"""
128+
Function pass the repo parameter to the "_extract_pullrequests" to fetch the pull requests of the particular repo.
129+
130+
Returns
131+
-------
132+
str
133+
return str saying that the file is stored if markdown is not empty.
134+
135+
"""
136+
dataframe = pd.DataFrame()
137+
for repo in self._list_of_repositories():
138+
dataframe = dataframe.append(self._extract_pullrequests(repo), ignore_index=True)
139+
140+
markdown = dataframe.to_markdown()
141+
142+
if len(markdown) > 0:
143+
# creating a markdown file
144+
# markdown_file = open(f"{self.filename}.md", "w")
145+
with open(f"{self.filename}.md", "w") as markdown_file:
146+
markdown_file.write(markdown)
147+
148+
return "Markdown File is successfully stored"
149+
150+
return "No pull requests found !!"
151+
152+
153+
if __name__ == "__main__":
154+
parser = argparse.ArgumentParser()
155+
parser.add_argument("-u", "--username", action="store_true")
156+
parser.add_argument("user", type=str, help="The name of the user to get the pull requests")
157+
parser.add_argument("-o", "--organization", action="store_true")
158+
parser.add_argument("organization_name", type=str, help="the organisation where user made the pull requests")
159+
parser.add_argument("-f", "--file", nargs="?")
160+
parser.add_argument("filename", type=str, nargs="?", help="filename to store the markdown table")
161+
args = parser.parse_args()
162+
if args.filename:
163+
file_name = args.filename
164+
else:
165+
file_name = "Markdown_file"
166+
if args.username and args.organization:
167+
response = Fetch_PullRequests(args.user, args.organization_name, file_name)
168+
print(response.get_pullrequests())
169+
else:
170+
print("Please pass atleast two arguments: '--username', '--organisation'")

‎Fetch_Contributions/requirements.txt‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
beautifulsoup4==4.9.3
2+
bs4==0.0.1
3+
certifi==2021年5月30日
4+
chardet==4.0.0
5+
idna==2.10
6+
lxml==4.6.3
7+
numpy==1.21.0
8+
pandas==1.3.0
9+
python-dateutil==2.8.1
10+
pytz==2021.1
11+
requests==2.25.1
12+
six==1.16.0
13+
soupsieve==2.2.1
14+
tabulate==0.8.9
15+
urllib3==1.26.6

‎SCRIPTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@
109109
| 107\. | ISBN Number Validator Script | Full form of ISBN number - International Standard Book Numbers. This python script can verify the entered number is valid isbn number or not. | [Take Me](./ISBN-Number-Validator/) | [Avdhesh Varshney](https://github.com/Avdhesh-Varshney)
110110
| 107\. | Image Classification | This script performs image classification using the MobileNetV2 pre-trained model provided by TensorFlow. It allows you to classify images into different categories based on the ImageNet dataset. | [Take Me](https://github.com/avinashkranjan/Amazing-Python-Scripts/tree/master/Image\Classification) | [Srujana Vanka](https://github.com/srujana-16)
111111
| 108\. | Check External IP Address | This is a basic Python script for determining your external IP address. | [Take Me](./Check_External_IP) | [Sabhi Sharma](https//github.com/sabhisharma-ise)
112+
| 109\. | Fetch Contributions | This script is a Python tool that fetches pull requests made by a user in a GitHub organization. | [Take Me](./Fetch_Contributions/) | [Sabhi Sharma](https//github.com/sabhisharma-ise)
112113
| 109\. | Domain Name Availability Checker | This script is a Python tool that allows you to check the availability of domain names using the GoDaddy API. | [Take Me](./Domain_Name_Availability/) | [Sabhi Sharma](https//github.com/sabhisharma-ise)
113114
| 110\. | Automatic Spelling Checker and Corrector | This Script is used to detect spelling errors in a text and correct them if the user wishes to do so. | [Take Me](./Automatic_Spelling_Checker_Corrector/) | [Sabhi Sharma](https//github.com/sabhisharma-ise)
114115
| 111\. | File Searcher | The File Search script is a Python tool that allows you to search for files with a specific extension in a directory. It recursively searches through all subdirectories of the specified directory and returns a list of files that match the provided file extension. | [Take Me](https://github.com/avinashkranjan/Amazing-Python-Scripts/tree/master/File\Search) | [Srujana Vanka](https://github.com/srujana-16)
115-

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /