Repository layer with multiple data sources

Question

I am implementing a repository layer in my mobile application. I would like the repository layer to complete to abstract the details about where the data it coming from/ or how we retrieve it from the service layer.

I have three data sources in my application

1. Data from a remote API
2. Data from a local SQLite DB
3. Data from a local In-memory cache

And I would like to access data using the following priority

1. Remote Only
2. Local Only
3. Remote First
4. Local First

I am using a generic repository pattern with Unit of work and it works great to handle all the DB transactions. But when I bring in the other 2 data-sources into the picture I will a bit confused on how to structure the code.

public interface IGenericRepository<TEntity> where TEntity : class, new()
{
 Task<TEntity> Get(string id);
 Task<IEnumerable<TEntity>> GetAll();
 Task<IEnumerable<TEntity>> Find<TValue>(Expression<Func<TEntity, bool>> predicate);
 Task<int> Add(TEntity entity);
 Task<int> AddRange(IEnumerable<TEntity> entities);
 Task<int> Remove(TEntity entity);
 Task<int> RemoveRange(IEnumerable<TEntity> entities);
 Task<int> UpdateItem(TEntity entity);
 Task<int> UpdateAllItems(IEnumerable<TEntity> entities);
}

Here is the implementation of the interface

public class GenericRepository<TEntity> : IGenericRepository<TEntity> where TEntity : class, new()
{
 private Lazy<IEncryptedDBConnection> _lazyDBConnection;
 protected IEncryptedDBConnection _dbConnection => _lazyDBConnection.Value;
 public GenericRepository(Lazy<IEncryptedDBConnection> lazyDBConnection)
 {
 _lazyDBConnection = lazyDBConnection;
 }
 public virtual async Task<TEntity> Get(string id)
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().FindAsync<TEntity>(id)).ConfigureAwait(false);
 }
 public virtual async Task<IEnumerable<TEntity>> GetAll()
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().Table<TEntity>().ToListAsync()).ConfigureAwait(false);
 }
 public virtual async Task<IEnumerable<TEntity>> Find<TValue>(Expression<Func<TEntity, bool>> predicate)
 {
 return await AttemptAndRetry(async () =>
 {
 var query = _dbConnection.GetAsyncConnection().Table<TEntity>();
 if (predicate != null)
 query = query.Where(predicate);
 return await query.ToListAsync();
 }).ConfigureAwait(false);
 }
 public virtual async Task<int> Add(TEntity entity)
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().InsertAsync(entity)).ConfigureAwait(false);
 }
 public virtual async Task<int> AddRange(IEnumerable<TEntity> entities)
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().InsertAllAsync(entities)).ConfigureAwait(false);
 }
 public virtual async Task<int> Remove(TEntity entity)
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().DeleteAsync(entity)).ConfigureAwait(false);
 }
 public virtual async Task<int> RemoveRange(IEnumerable<TEntity> entities)
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().DeleteAllAsync<TEntity>()).ConfigureAwait(false);
 }
 public virtual async Task<int> UpdateItem(TEntity entity)
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().UpdateAsync(entity)).ConfigureAwait(false);
 }
 public virtual async Task<int> UpdateAllItems(IEnumerable<TEntity> entities)
 {
 return await AttemptAndRetry(() => _dbConnection.GetAsyncConnection().UpdateAllAsync(entities)).ConfigureAwait(false);
 }
 protected Task<TResult> AttemptAndRetry<TResult>(Func<Task<TResult>> action, int numRetries = 3)
 {
 return Policy.Handle<SQLiteException>().WaitAndRetryAsync(numRetries, PollyRetryAttempt).ExecuteAsync(action);
 }
 private TimeSpan PollyRetryAttempt(int attemptNumber) => TimeSpan.FromMilliseconds(Math.Pow(2, attemptNumber));
}

I also have various other classes that inherit GenericRepsotiory to make any additonal chnages such as ProductRepository, OrderRepository etc.

My Questions

• Which is the right class to add the logic to choose the data access source whether remote only or local only etc?
• Do I add the remote API calls into the repository classes or do I write other classes to make the actual calls?

• Peter Csala: Are you aware of the fact that whenever you have multiple data manipulation logic against different data sources then you have entered into the realm of distributed transactions (and saga)?
• Pedro Rodrigues: @PeterCsala Not necessarily. One could be a local cache, for example. Or am I missing something?
• Peter Csala: @PedroRodrigues In case of local cache what you really want is to have a one-way sync-ed read-only lookup table. If you want to write that data as well then you need two-way synchronization which would highly increase the complexity and can introduce merge conflicts. So, if you have to write a database and any other datastore (a file, ftp server, nosql database, whatever) in order to preserve data consistency there should be an external coordinator (call it DTM or Saga orchestrator) between the parties.

Answer 1

Have n + 1 implementations of IGenericRepository for every type, where n is the number of datasources.

n because you will implement every datasource separatly. + 1 because you need a mediator between the various sources.

Keep responsabilities separate. Separate responsability chaining and data access.

• Libin Joseph: Is there a specfic name for that pattern to be the mediator. I am using a strategy pattern at the moment and seems to work. But would be keen to know your thoughts
• Pedro Rodrigues: that really depends on the actual use case. The OP example seems to be a good fit for the mediator pattern, as you point out. But others can apply as well, facade, chain of rssponsability.
• Farid: To be honest, this makes no sense. If you were to create a repository per data source, what is the purpose of the repository here, anyway?
• Pedro Rodrigues: @Farid I wasn't quite explicit there I believe. One interface for the repository, one implementation of said interface per datasource (MSSQL, MySQL, etc), and a facade that abstracts implementation selection. The repository serves as the point of contact to the application, decoupling it from the actual implementation.
• Farid: Well they are valid candidates for data sources why treat them as repositories?