From 9535100c85abc3426c60f9beea66a1235a87680a Mon Sep 17 00:00:00 2001 From: KernelDeimos Date: Thu, 17 Oct 2024 03:00:35 -0400 Subject: [PATCH] doc: add brainstorming for mountpoints --- doc/devlog.md | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 doc/devlog.md diff --git a/doc/devlog.md b/doc/devlog.md new file mode 100644 index 00000000..3fc83773 --- /dev/null +++ b/doc/devlog.md @@ -0,0 +1,103 @@ +## 2024-10-16 + +### Considerations for Mountpoints Feature + +- `_storage_upload` takes paramter `uuid` instead of `path` + - S3 bucket strategy needs the UUID + - If we do hashes, 10MB chunks should be fine + - we're already able to smooth out bursty traffic using the + EWA algorithm +- Use of `systemFSEntryService` + - Is that normalized? Does everything go through this interface? +- Storage interface has methods like `post_insert` + - as far as I can tell this doesn't pose any issue +- + +### Brainstorming Migration Strategies + +#### Interface boundary at HL<->LL filesystem methods + +-- **tags:** brainstorming + +From the perspectice of a trait-oriented implementation, +which is not how LL/HL filesystem operations are currently implemented, +the LL-class operations are implemented in separate traits. + +The composite trait containing all of these traits would be the trait +that represents a filesystem implementation itself. + +Other filesystem interfaces that I've seen, such as FUSE and 9p, +all usually have a monolithic interface - that is to say, an interface +which includes all of the filesystem operations, rather than several +interfaces each implementing a single filesystem operaiton. + +Something about the fact that the LL-class operations are in separate +classes makes it difficult to reason about how to move. +Is it simply that multiple files in a directory is just more +annoying to think about? Maybe, but there must be something more. + +Perhaps it's that there are several references. Each implementation +(that is, implemenation of a single filesystem operation) could have +any number of different references across any number of different files. +This would not be the case with a monolithic interface. + +I think the best of both worlds would be to have an interface representing +the entire filesystem and, in one place, link of of the individual +operation implementations to compose a filesystem implementation + +### Filesystem Brainstorming + +Puter's backend uses a service architecture. Each service is an instance +of a class extending "Service". A service can listen to events of the +backend's lifecycle, interact with other services, and interact with +external interfaces such as APIs and databases. + +Puter's current filesystem, let's call it PuterFSv1, exists as the result +of multiple services working together. We have LocalDiskStorageService +which mimics an S3 bucket on a local system, and we have +DatabaseFSEntryService which manages information about files, directories, +and their relationships within the database, and therefore depends on +DatabaseAccessService. + +It is now time to introduce a MountpointService. This will allow another +service or a user's configuration to assign an instance of a filesystem +implementation (such as PuterFSv1) to a specific path. + +The trouble here is that PuterFSv1 is composed of services, and the nature +of a service is such that it exists for the lifecycle of the application. +The class for a particular service can be re-used and registered with +multiple names (creating multiple services with the same implementation +but perhaps different configuration), but that's only a clean scenario when +there is just one service. PuterFSv1, on the other hand, is like an +imaginary service composed of other services. + +The following possibilities then should be discussed: +- CompositeService base class for a service that is composed of + more than one service. +- Refactor filesystem to not use service architecture. +- Each filesystem service can manage state and configuration + for multiple mountpoints + (I don't like this idea; it feels messy. I wonder what software + principles this violates) + +We can take advantage of traits/interfaces here. +PuterFSv1 depends on two interfaces: +- An S3-like data storage implementation +- An fsentry storage implementation + +Counterintuitively from what I first thought, "Refactor the filesystem" +actually looks like the best solution, and it doens't even look like it +will be that difficult. In fact, it'll likely make the filesystem easier +to maintain and more robust as a result. + +Additionally, we can introduce PuterFSv2, which will introduce storing +data in chunks identified by their hashes, and associated hashes with +fsentries. + +PuterFSService will be a new service which registers 'PuterFSv1' with +FilesystemService. + +An instance of a filesystem needs to be separate from a mountpoint. +For example, PuterFSv1 will usually have only one instance but it may +be mounted several different times. `/some-user` on Puter's VFS could +be a mountpoint for `/some-user` in the instance of PuterFSv1.