Status quo
Currently the get_queryset simply adopted the defined prefetch_related and select_related definitions. But there are still some existing bottlenecks:

Scenario: No include parameter
Given is: The used serializer has a ResourceRelatedField and no include parameter is used.
For that case no predefinied prefetch_related definition is used.
But the ResourceRelatedField needs all ids from the relation, so it will trigger the db to get it.

Solution: No include paramter
My current solution is to define conditional prefetch_related definitions inside the get_queryset() on my views.
Something like

def get_queryset(self):
 qs = super().get_queryset()
 include = self.request.GET.get("include", None)
 if not include or "fieldName" not in include:
 qs = qs.prefetch_related(Prefetch("field_name", queryset=Model.objects.only('id'))
 return qs

Without the prefetch_related definition it leads into the described m*(n+1) problems

But there is another point. The code below prefetches in any case. But if the api is called with sparse fieldsets and the related field is not part of the request, the prefetching is not needed.

So it need another logic to get the fields parameters and analyze them against the prefetching logic. Something like:

fields = self.request.GET.get("fields[ResourceName]", "").split(',')
if fields and "fieldName" in fields:
 # do prefetching
else:
 # do not add default prefetching

So i think this is a general problem which hits every view with a ResourceRelatedField, for that it would be worth it to add some logic to the core package to handle this.