Paginating API with "next" Pythonically

Question

I'm wondering if there's a more Pythonic way to keep paginating an API endpoint until there's no next key in the response. The thing that bothers me the most is the for item in json_resp.get("data") (I know this can be extracted as a function).

# some code is omitted, some obfuscated
# attachment of token in API calls omitted for code brevity
def get_object_list_for_user(user_service_token): 
 endpoint = f"{HOSTNAME}/me/library/objects/"
 params = {"limit": 100}
 response = _call_service_api(endpoint, params=params)
 json_resp = response.json()
 lists_of_objects_to_append = []
 # get first batch
 for item in json_resp.get("data"):
 object_dict = {"library_id": item["id"], "name": item["attributes"]["name"]}
 lists_of_objects_to_append.append(object_dict)
 offset = 0
 while json_resp.get("next"):
 offset += 100
 params.update({"offset": offset})
 response = _call_service_api(endpoint, params=params)
 json_resp = response.json()
 for item in json_resp.get("data"):
 object_dict = {"library_id": item["id"], "name": item["attributes"]["name"]}
 lists_of_objects_to_append.append(object_dict)
 return lists_of_objects_to_append

Answer 1

I attempted to reduce redundancy by simulating a do-while loop so that the first condition initializes the sequence and then the code can be factored out.

Instead of having a temporary list, it returns a generator which can be appended to the list.

I couldn't test this so I don't know if it works:

def get_object_list_for_user(user_service_token): 
 endpoint = f"{HOSTNAME}/me/library/objects/"
 params = {"limit": 100}
 offset = 0
 lists_of_objects_to_append = []
 while True:
 json_resp = _call_service_api(endpoint, params=params).json()
 lists_of_objects_to_append.extend(extract_items(json_resp))
 offset += 100
 params.update({"offset": offset})
 if not json_resp.get("next"):
 return lists_of_objects_to_append
def extract_items(json_resp):
 for item in json_resp.get("data"):
 yield({"library_id": item["id"], "name": item["attributes"]["name"]})