Multiple file access abstractions

Question

When using dependency injection for nearly everything it's good to have some file access abstraction. I find the idea of ASP.NET Core FileProvider nice but not sufficient for my needs so inspired by that I decided to create my own with some more functionality.

Follow-up

---

Interfaces

I have two interfaces that are called just like theirs but they have different members and also other names.

The first interface represents a single file or a directory.

[PublicAPI]
public interface IFileInfo : IEquatable<IFileInfo>, IEquatable<string>
{
 [NotNull]
 string Path { get; }
 [NotNull]
 string Name { get; }
 bool Exists { get; }
 long Length { get; }
 DateTime ModifiedOn { get; } 
 bool IsDirectory { get; }
 [NotNull]
 Stream CreateReadStream();
}

The other interface allows me to perform four of the basic file/directory operations:

[PublicAPI]
public interface IFileProvider
{
 [NotNull]
 IFileInfo GetFileInfo([NotNull] string path);
 [NotNull]
 IFileInfo CreateDirectory([NotNull] string path);
 [NotNull]
 IFileInfo DeleteDirectory([NotNull] string path, bool recursive);
 [NotNull]
 Task<IFileInfo> CreateFileAsync([NotNull] string path, [NotNull] Stream data);
 [NotNull]
 IFileInfo DeleteFile([NotNull] string path);
}

---

On top of them I've build three providers:

• PhysicalFileProvider and PhysicalFileInfo - used for operation on the physical drive
• EmbeddedFileProvider and EmbeddedFileInfo - used for reading of embedded resources (primarily for testing); internally, it automatically adds the root namespace of the specified assembly to the path
• InMemoryFileProvider and InMemoryFileInfo - used for testing or runtime data

There is no GetDirectoryContents API because this is what I have the DirectoryTree for.

All providers use the same path schema, this is, with the backslash \. This is also why the EmbeddedFileProvider does some additional converting between the usual patch and the resource path which is separated by dots .

---

Implementations

So here they are, the three pairs, in the same order as the above list:

[PublicAPI]
public class PhysicalFileProvider : IFileProvider
{
 public IFileInfo GetFileInfo(string path)
 {
 if (path == null) throw new ArgumentNullException(nameof(path));
 return new PhysicalFileInfo(path);
 }
 public IFileInfo CreateDirectory(string path)
 {
 if (path == null) throw new ArgumentNullException(nameof(path));
 if (Directory.Exists(path))
 {
 return new PhysicalFileInfo(path);
 }
 try
 {
 var newDirectory = Directory.CreateDirectory(path);
 return new PhysicalFileInfo(newDirectory.FullName);
 }
 catch (Exception ex)
 {
 throw new CreateDirectoryException(path, ex);
 }
 }
 public async Task<IFileInfo> CreateFileAsync(string path, Stream data)
 {
 try
 {
 using (var fileStream = new FileStream(path, FileMode.CreateNew, FileAccess.Write))
 {
 await data.CopyToAsync(fileStream);
 await fileStream.FlushAsync();
 }
 return new PhysicalFileInfo(path);
 }
 catch (Exception ex)
 {
 throw new CreateFileException(path, ex);
 }
 }
 public IFileInfo DeleteFile(string path)
 {
 if (path == null) throw new ArgumentNullException(nameof(path));
 try
 {
 File.Delete(path);
 return new PhysicalFileInfo(path);
 }
 catch (Exception ex)
 {
 throw new DeleteFileException(path, ex);
 }
 }
 public IFileInfo DeleteDirectory(string path, bool recursive)
 {
 try
 {
 Directory.Delete(path, recursive);
 return new PhysicalFileInfo(path);
 }
 catch (Exception ex)
 {
 throw new DeleteDirectoryException(path, ex);
 }
 }
}
[PublicAPI]
internal class PhysicalFileInfo : IFileInfo
{
 public PhysicalFileInfo([NotNull] string path) => Path = path ?? throw new ArgumentNullException(nameof(path));
 #region IFileInfo
 public string Path { get; }
 public string Name => System.IO.Path.GetFileName(Path);
 public bool Exists => File.Exists(Path) || Directory.Exists(Path);
 public long Length => Exists && !IsDirectory ? new FileInfo(Path).Length : -1;
 public DateTime ModifiedOn => !string.IsNullOrEmpty(Path) ? File.GetLastWriteTime(Path) : default;
 public bool IsDirectory => Directory.Exists(Path);
 public Stream CreateReadStream()
 {
 return
 IsDirectory
 ? throw new InvalidOperationException($"Cannot open '{Path}' for reading because it's a directory.")
 : Exists
 ? File.OpenRead(Path)
 : throw new InvalidOperationException("Cannot open '{Path}' for reading because the file does not exist.");
 }
 #endregion
 #region IEquatable<IFileInfo>
 public override bool Equals(object obj) => obj is IFileInfo file && Equals(file);
 public bool Equals(IFileInfo other) => FileInfoEqualityComparer.Default.Equals(other, this);
 public bool Equals(string other) => FileInfoEqualityComparer.Default.Equals(other, Path);
 public override int GetHashCode() => FileInfoEqualityComparer.Default.GetHashCode(this);
 #endregion
}

---

public class EmbeddedFileProvider : IFileProvider
{
 private readonly Assembly _assembly;
 public EmbeddedFileProvider([NotNull] Assembly assembly)
 {
 _assembly = assembly ?? throw new ArgumentNullException(nameof(assembly));
 BasePath = _assembly.GetName().Name.Replace('.', '\\');
 }
 public string BasePath { get; }
 public IFileInfo GetFileInfo(string path)
 {
 if (path == null) throw new ArgumentNullException(nameof(path));
 // Embedded resouce names are separated by '.' so replace the windows separator.
 var fullName = Path.Combine(BasePath, path).Replace('\\', '.');
 // Embedded resource names are case sensitive so find the actual name of the resource.
 var actualName = _assembly.GetManifestResourceNames().FirstOrDefault(name => SoftString.Comparer.Equals(name, fullName));
 var getManifestResourceStream = actualName is null ? default(Func<Stream>) : () => _assembly.GetManifestResourceStream(actualName);
 return new EmbeddedFileInfo(UndoConvertPath(fullName), getManifestResourceStream);
 }
 // Convert path back to windows format but the last '.' - this is the file extension.
 private static string UndoConvertPath(string path) => Regex.Replace(path, @"\.(?=.*?\.)", "\\");
 public IFileInfo CreateDirectory(string path)
 {
 throw new NotSupportedException($"{nameof(EmbeddedFileProvider)} does not support directory creation.");
 }
 public IFileInfo DeleteDirectory(string path, bool recursive)
 {
 throw new NotSupportedException($"{nameof(EmbeddedFileProvider)} does not support directory deletion.");
 }
 public Task<IFileInfo> CreateFileAsync(string path, Stream data)
 {
 throw new NotSupportedException($"{nameof(EmbeddedFileProvider)} does not support file creation.");
 }
 public IFileInfo DeleteFile(string path)
 {
 throw new NotSupportedException($"{nameof(EmbeddedFileProvider)} does not support file deletion.");
 }
}
internal class EmbeddedFileInfo : IFileInfo
{
 private readonly Func<Stream> _getManifestResourceStream;
 public EmbeddedFileInfo(string path, Func<Stream> getManifestResourceStream)
 {
 _getManifestResourceStream = getManifestResourceStream;
 Path = path;
 }
 public string Path { get; }
 public string Name => System.IO.Path.GetFileNameWithoutExtension(Path);
 public bool Exists => !(_getManifestResourceStream is null);
 public long Length => _getManifestResourceStream()?.Length ?? -1;
 public DateTime ModifiedOn { get; }
 public bool IsDirectory => false;
 // No protection necessary because there are no embedded directories.
 public Stream CreateReadStream() => _getManifestResourceStream();
 #region IEquatable<IFileInfo>
 public override bool Equals(object obj) => obj is IFileInfo file && Equals(file);
 public bool Equals(IFileInfo other) => FileInfoEqualityComparer.Default.Equals(other, this);
 public bool Equals(string other) => FileInfoEqualityComparer.Default.Equals(other, Path);
 public override int GetHashCode() => FileInfoEqualityComparer.Default.GetHashCode(this);
 #endregion
}

---

public class InMemoryFileProvider : Dictionary<string, byte[]>, IFileProvider
{
 private readonly ISet<IFileInfo> _files = new HashSet<IFileInfo>();
 #region IFileProvider
 public IFileInfo GetFileInfo(string path)
 {
 var file = _files.SingleOrDefault(f => FileInfoEqualityComparer.Default.Equals(f.Path, path));
 return file ?? new InMemoryFileInfo(path, default(byte[]));
 }
 public IFileInfo CreateDirectory(string path)
 {
 path = path.TrimEnd('\\');
 var newDirectory = new InMemoryFileInfo(path, _files.Where(f => f.Path.StartsWith(path)));
 _files.Add(newDirectory);
 return newDirectory;
 }
 public IFileInfo DeleteDirectory(string path, bool recursive)
 {
 return DeleteFile(path);
 }
 public Task<IFileInfo> CreateFileAsync(string path, Stream data)
 {
 var file = new InMemoryFileInfo(path, GetByteArray(data));
 _files.Remove(file);
 _files.Add(file);
 return Task.FromResult<IFileInfo>(file);
 byte[] GetByteArray(Stream stream)
 {
 using (var memoryStream = new MemoryStream())
 {
 stream.CopyTo(memoryStream);
 return memoryStream.ToArray();
 }
 }
 }
 public IFileInfo DeleteFile(string path)
 {
 var fileToDelete = new InMemoryFileInfo(path, default(byte[]));
 _files.Remove(fileToDelete);
 return fileToDelete;
 }
 #endregion
}
internal class InMemoryFileInfo : IFileInfo
{
 [CanBeNull]
 private readonly byte[] _data;
 [CanBeNull]
 private readonly IEnumerable<IFileInfo> _files;
 private InMemoryFileInfo([NotNull] string path)
 {
 Path = path ?? throw new ArgumentNullException(nameof(path));
 ModifiedOn = DateTime.UtcNow;
 }
 public InMemoryFileInfo([NotNull] string path, byte[] data)
 : this(path)
 {
 _data = data;
 Exists = !(data is null);
 IsDirectory = false;
 }
 public InMemoryFileInfo([NotNull] string path, [NotNull] IEnumerable<IFileInfo> files)
 : this(path)
 {
 _files = files ?? throw new ArgumentNullException(nameof(files));
 Exists = true;
 IsDirectory = true;
 }
 #region IFileInfo
 public bool Exists { get; }
 public long Length => IsDirectory ? throw new InvalidOperationException("Directories have no length.") : _data?.Length ?? -1;
 public string Path { get; }
 public string Name => System.IO.Path.GetFileNameWithoutExtension(Path);
 public DateTime ModifiedOn { get; }
 public bool IsDirectory { get; }
 public Stream CreateReadStream()
 {
 return
 IsDirectory
 ? throw new InvalidOperationException("Cannot create read-stream for a directory.")
 : Exists
 // ReSharper disable once AssignNullToNotNullAttribute - this is never null because it's protected by Exists.
 ? new MemoryStream(_data)
 : throw new InvalidOperationException("Cannot create a read-stream for a file that does not exist.");
 }
 #endregion
 #region IEquatable<IFileInfo>
 public override bool Equals(object obj) => obj is IFileInfo file && Equals(file);
 public bool Equals(IFileInfo other) => FileInfoEqualityComparer.Default.Equals(other, this);
 public bool Equals(string other) => FileInfoEqualityComparer.Default.Equals(other, Path);
 public override int GetHashCode() => FileInfoEqualityComparer.Default.GetHashCode(this);
 #endregion
}

---

Decorator for less typing

There is one more file provider. I use this to save some typing of paths. The RelativeFileProvider adds its path in front of the other path if there is some root path that doesn't change.

public class RelativeFileProvider : IFileProvider
{
 private readonly IFileProvider _fileProvider;
 private readonly string _basePath;
 public RelativeFileProvider([NotNull] IFileProvider fileProvider, [NotNull] string basePath)
 {
 _fileProvider = fileProvider ?? throw new ArgumentNullException(nameof(fileProvider));
 _basePath = basePath ?? throw new ArgumentNullException(nameof(basePath));
 }
 public IFileInfo GetFileInfo(string path) => _fileProvider.GetFileInfo(CreateFullPath(path));
 public IFileInfo CreateDirectory(string path) => _fileProvider.CreateDirectory(CreateFullPath(path));
 public IFileInfo DeleteDirectory(string path, bool recursive) => _fileProvider.DeleteDirectory(CreateFullPath(path), recursive);
 public Task<IFileInfo> CreateFileAsync(string path, Stream data) => _fileProvider.CreateFileAsync(CreateFullPath(path), data);
 public IFileInfo DeleteFile(string path) => _fileProvider.DeleteFile(CreateFullPath(path));
 private string CreateFullPath(string path) => Path.Combine(_basePath, path ?? throw new ArgumentNullException(nameof(path)));
}

---

Exceptions

The providers don't throw pure .NET exception because they aren't usually helpful. I wrap them in my own types:

public class CreateDirectoryException : Exception
{
 public CreateDirectoryException(string path, Exception innerException)
 : base($"Could not create directory: {path}", innerException)
 { }
}
public class CreateFileException : Exception
{
 public CreateFileException(string path, Exception innerException)
 : base($"Could not create file: {path}", innerException)
 { }
}
public class DeleteDirectoryException : Exception
{
 public DeleteDirectoryException(string path, Exception innerException)
 : base($"Could not delete directory: {path}", innerException)
 { }
}
public class DeleteFileException : Exception
{
 public DeleteFileException(string path, Exception innerException)
 : base($"Could not delete file: {path}", innerException)
 { }
}

---

Simple file search

There is one more provider that allows me to probe multiple providers. It supports only reading:

public class CompositeFileProvider : IFileProvider
{
 private readonly IEnumerable<IFileProvider> _fileProviders;
 public CompositeFileProvider(IEnumerable<IFileProvider> fileProviders)
 {
 _fileProviders = fileProviders;
 }
 public IFileInfo GetFileInfo(string path)
 {
 foreach (var fileProvider in _fileProviders)
 {
 var fileInfo = fileProvider.GetFileInfo(path);
 if (fileInfo.Exists)
 {
 return fileInfo;
 }
 }
 return new InMemoryFileInfo(path, new byte[0]);
 }
 public IFileInfo CreateDirectory(string path)
 {
 throw new NotSupportedException($"{nameof(CompositeFileProvider)} does not support directory creation.");
 }
 public IFileInfo DeleteDirectory(string path, bool recursive)
 {
 throw new NotSupportedException($"{nameof(CompositeFileProvider)} does not support directory deletion.");
 }
 public Task<IFileInfo> CreateFileAsync(string path, Stream data)
 {
 throw new NotSupportedException($"{nameof(CompositeFileProvider)} does not support file creation.");
 }
 public IFileInfo DeleteFile(string path)
 {
 throw new NotSupportedException($"{nameof(CompositeFileProvider)} does not support file deletion.");
 }
}

---

Comparing files

The comparer for the IFileInfo is very straightforward and compares the Path property:

public class FileInfoEqualityComparer : IEqualityComparer<IFileInfo>, IEqualityComparer<string>
{
 private static readonly IEqualityComparer PathComparer = StringComparer.OrdinalIgnoreCase;
 [NotNull]
 public static FileInfoEqualityComparer Default { get; } = new FileInfoEqualityComparer();
 public bool Equals(IFileInfo x, IFileInfo y) => Equals(x?.Path, y?.Path);
 public int GetHashCode(IFileInfo obj) => GetHashCode(obj.Path);
 public bool Equals(string x, string y) => PathComparer.Equals(x, y);
 public int GetHashCode(string obj) => PathComparer.GetHashCode(obj);
}

---

Example

As an example I use one of my tests that checks whether the relative and embedded files providers do their job correctly.

[TestClass]
public class RelativeFileProviderTest
{
 [TestMethod]
 public void GetFileInfo_DoesNotGetNonExistingEmbeddedFile()
 {
 var fileProvider =
 new RelativeFileProvider(
 new EmbeddedFileProvider(typeof(RelativeFileProviderTest).Assembly),
 @"relative\path");
 var file = fileProvider.GetFileInfo(@"file.ext");
 Assert.IsFalse(file.Exists);
 Assert.IsTrue(SoftString.Comparer.Equals(@"Reusable\Tests\relative\path\file.ext", file.Path)); 
 }
}

---

Questions

Besides of the default question about can this be improved in anyway I have more:

• should I be concerned about thread-safty here? I didn't use any locks but adding them isn't a big deal. Should I? Where would you add them? I guess creating files and directories could be good candidates, right?
• should the EmbeddedFileProvider use the RelativeFileProvider to add the assembly namespace to the path or should I leave it as is?

• benj2240: Did you consider System.IO.Abstractions? I like using that one.
• t3chb0t: @benj2240 I saw this one but it hasn't the functionality I need ;-)
• Bent Rasmussen: APIs do not separate out the concern of mutation, such that all file providers enable write access, at least on a type level. Expressing either readable or writable files in C# may not be that elegant; but one can image several file systems that are not writable. Not sure what your concrete needs are though.

Answer 1

As usual there isn't much left to say about your code then I'll try to imagine to use it:

I'd expect CreateFileAsync() to only create the file like File.Create(), returning a Stream to that (abstract) file. It might be even easier to use because instead of:

using (var stream = new MemoryStream()) {
 stream.Write(...);
 await output.CreateFileAsync(path, stream);
}

I might write:

using (var stream = await output.CreateFile(path)) {
 stream.Write(...); 
}

There is something similar for reading (in IFileInfo) then it might be handy to have it also for writing. Not a big deal if I can't but then I'd probably expect CreateFileAsync() to be named CreateFileFromStreamAsync() (or something like that).

---

I'd, personally, love to have an *Async() version of all those methods. This code might be easily extended to work through a network using different transports. What about a (finally) easy to use FTP client? I might not want to wait for a Delete() call to complete (or I might want to do something else while waiting, maybe preparing the file to upload).

---

EmbeddedFileProvider expects \ as directory separator, you might use Path.DirectorySeparatorChar instead of the hard-coded character and you should probably check also for Path.AltDirectorySeparatorChar. This code might be compiled for .NET Standard and run on .NET Core on a Linux machine. Even if Windows is the only target I appreciate applications which handle both when I switch often from Linux to Windows (and I suppose I'm not the only one).

---

I'd like to have few read-only properties to check what a IFileInfo supports: CanRead, CanWrite, CanDelete and similar. As a caller I might find easier to check for that property instead of catching an exception (that's toooo Pythonic).

---

should I be concerned about thread-safty here? I didn't use any locks but adding them isn't a big deal. Should I? Where would you add them? I guess creating files and directories could be good candidates, right?

I suppose you should not. .NET streams are not thread-safe, native file system is thread-safe in the sense that its behavior is well-defined then - for consistency - I'd at least change DeleteFile() and DeleteDirectory() of InMemoryFileProvider to mimic the same behavior (assuming FileShare.Delete if file is "in use" it'll be deleted when every handle is closed). Of course to do this you have to add locks to its functions (and if IFileInfo is not IDisposable current implementation works pretty smoothly).

Can you do more? Probably but we will go in the opinions realm and I appreciate foundation classes to be as unopinionated as possible.

should the EmbeddedFileProvider use the RelativeFileProvider to add the assembly namespace to the path or should I leave it as is?

I'd leave it as-is but it's more a gut feeling than an educated decision. I don't think it can make code better or easier to understand.

• t3chb0t: These are some great suggestions! I actually was considering to create an ftp-file-provider later so making all methods *Async is definitely a good idea. I also like the helper Can* properties very much.
• Adriano Repetti: @t3chb0t when you will be done with HTTP and FTP implementations and if this code can be open sourced...you should create a nuget package. It can be extended to compound files (FAT32 on file? OLE Compound Files?) and I never saw a good ground to glue all of them.
• t3chb0t: I wasn't planning to create another package for it but it's part of my Reusable.Core package and the source code can be found here. I'll post a follow-up when I'm done with the FTP and share-file-provider because I might be needing some kind of a session for it... it's still on a drawing-board ;-)
• t3chb0t: I was just implementing your suggestions and found that the DirectorySeparatorChar is a member of Path and not Directory... I've fixed it in your answer.
• Adriano Repetti: Even more interesting...I thought it may just wrap path into Environment.ExpandEnvironmentVariables() but if the list (or the syntax...) is customizable then even better! Yep links are tricky, you might just add an overload with a Predicate<string> callback and let the caller deal with them (if she has to...) but it doesn't happen often, probably a generic library won't need it that much

Answer 2

In PhysicalFileProvider.CreateDirectory() you should place the call to Directory.Exists() inside the try..catch as well because it can throw e.g ArgumentException for "C:\test?" or a NotSupportedException for C:\:.

But basically you could skip this check at all because calling Directory.CreateDirectory() will do the check itself (Directory.CreateDiractory reference source).

In PhysicalFileInfo.ModifiedOn you should change the check from !string.IsNullOrEmpty(Path) to !string.IsNullOrWhiteSpace(Path) to avoid an ArgumentException if Path only contains whitespace characters.
Well basically you should validate the path in your ctor some more, e.g for illigal characters etc. to avoid your methods to throw exceptions.

Otherwise your code looks clean as usual and is easy to understand. At least PhysicalFileInfo and PhysicalFileProvider are thread-safe because you don't change any class level state outside of the ctor.

A small nitpick: Regions are smelling.

• t3chb0t: Regions are smelling. - this is the only case where I actually use regions, to group interface members or sometimes operator overloads, other than this I dislike them too - will you forgive me for that? ;-]
• Heslacher: Ok, but thats the last time ;-)

Answer 3

In for instance EmbeddedFileInfo I wonder if you can sharpen the first condition in:

public override bool Equals(object obj) => obj is IFileInfo file && Equals(file);

to:

public override bool Equals(object obj) => obj is EmbeddedFileInfo file && Equals(file);

But you only compare on path so maybe not, because?

---

I think I would implement PhysicalFileInfo in this way:

 internal class PhysicalFileInfo : IFileInfo
 {
 FileSystemInfo m_info;
 public PhysicalFileInfo([NotNull] string path)
 {
 m_info = File.Exists(path) ? 
 new FileInfo(path) as FileSystemInfo : 
 (Directory.Exists(path) ? new DirectoryInfo(path) : 
 throw new ArgumentException("Invalid Path", nameof(path)));
 }
 #region IFileInfo
 public string Path => m_info.FullName;
 public string Name => m_info.Name;
 public bool Exists => m_info.Exists;
 public long Length => Exists && !IsDirectory ? (m_info as FileInfo).Length : -1;
 public DateTime ModifiedOn => m_info.LastWriteTime;
 public bool IsDirectory => m_info is DirectoryInfo;
 public Stream CreateReadStream()
 {
 return
 IsDirectory
 ? throw new InvalidOperationException($"Cannot open '{Path}' for reading because it's a directory.")
 : Exists
 ? (m_info as FileInfo).OpenRead()
 : throw new InvalidOperationException("Cannot open '{Path}' for reading because the file does not exist.");
 }
 #endregion
 #region IEquatable<IFileInfo>
 public override bool Equals(object obj) => obj is IFileInfo file && Equals(file);
 public bool Equals(IFileInfo other) => FileInfoEqualityComparer.Default.Equals(other, this);
 public bool Equals(string other) => FileInfoEqualityComparer.Default.Equals(other, Path);
 public override int GetHashCode() => FileInfoEqualityComparer.Default.GetHashCode(this);
 #endregion
 }

---

In this:

public Stream CreateReadStream()
{
 return
 IsDirectory
 ? throw new InvalidOperationException($"Cannot open '{Path}' for reading because it's a directory.")
 : Exists
 ? File.OpenRead(Path)
 : throw new InvalidOperationException("Cannot open '{Path}' for reading because the file does not exist.");
}

you throw InvalidOperationException if Exists returns false. But if the file is deleted by another process between the call to Exists and File.OpenRead(Path) a FileNotFoundException will be thrown by the system. So there is a minor risk for two different error message for the same exception/error for the same operation. In general I would avoid checking if the file/directory exists before any operation on them and let the system respond with the standard exceptions. And if you find it necessary to provide your own exceptions then catch the standard exceptions in the method in question and the throw your own to the caller.

• t3chb0t: Exception handling is always the most tricky part and with time I'm becomming more and more defensive in the sense that I often throw too many exceptions than rely on the default ones. Standard exceptions are virtually never helpful ;-) They indicate some problem but never tell you what was really the cause. I like the idea with the FileInfo and how you check whether this is a directory; this is very clever ;-]