mirror of
https://github.com/HeyPuter/puter
synced 2024-11-13 21:52:23 +00:00
doc: add brainstorming for mountpoints
This commit is contained in:
parent
7980aafbd9
commit
9535100c85
103
doc/devlog.md
Normal file
103
doc/devlog.md
Normal file
@ -0,0 +1,103 @@
|
||||
## 2024-10-16
|
||||
|
||||
### Considerations for Mountpoints Feature
|
||||
|
||||
- `_storage_upload` takes paramter `uuid` instead of `path`
|
||||
- S3 bucket strategy needs the UUID
|
||||
- If we do hashes, 10MB chunks should be fine
|
||||
- we're already able to smooth out bursty traffic using the
|
||||
EWA algorithm
|
||||
- Use of `systemFSEntryService`
|
||||
- Is that normalized? Does everything go through this interface?
|
||||
- Storage interface has methods like `post_insert`
|
||||
- as far as I can tell this doesn't pose any issue
|
||||
-
|
||||
|
||||
### Brainstorming Migration Strategies
|
||||
|
||||
#### Interface boundary at HL<->LL filesystem methods
|
||||
|
||||
-- **tags:** brainstorming
|
||||
|
||||
From the perspectice of a trait-oriented implementation,
|
||||
which is not how LL/HL filesystem operations are currently implemented,
|
||||
the LL-class operations are implemented in separate traits.
|
||||
|
||||
The composite trait containing all of these traits would be the trait
|
||||
that represents a filesystem implementation itself.
|
||||
|
||||
Other filesystem interfaces that I've seen, such as FUSE and 9p,
|
||||
all usually have a monolithic interface - that is to say, an interface
|
||||
which includes all of the filesystem operations, rather than several
|
||||
interfaces each implementing a single filesystem operaiton.
|
||||
|
||||
Something about the fact that the LL-class operations are in separate
|
||||
classes makes it difficult to reason about how to move.
|
||||
Is it simply that multiple files in a directory is just more
|
||||
annoying to think about? Maybe, but there must be something more.
|
||||
|
||||
Perhaps it's that there are several references. Each implementation
|
||||
(that is, implemenation of a single filesystem operation) could have
|
||||
any number of different references across any number of different files.
|
||||
This would not be the case with a monolithic interface.
|
||||
|
||||
I think the best of both worlds would be to have an interface representing
|
||||
the entire filesystem and, in one place, link of of the individual
|
||||
operation implementations to compose a filesystem implementation
|
||||
|
||||
### Filesystem Brainstorming
|
||||
|
||||
Puter's backend uses a service architecture. Each service is an instance
|
||||
of a class extending "Service". A service can listen to events of the
|
||||
backend's lifecycle, interact with other services, and interact with
|
||||
external interfaces such as APIs and databases.
|
||||
|
||||
Puter's current filesystem, let's call it PuterFSv1, exists as the result
|
||||
of multiple services working together. We have LocalDiskStorageService
|
||||
which mimics an S3 bucket on a local system, and we have
|
||||
DatabaseFSEntryService which manages information about files, directories,
|
||||
and their relationships within the database, and therefore depends on
|
||||
DatabaseAccessService.
|
||||
|
||||
It is now time to introduce a MountpointService. This will allow another
|
||||
service or a user's configuration to assign an instance of a filesystem
|
||||
implementation (such as PuterFSv1) to a specific path.
|
||||
|
||||
The trouble here is that PuterFSv1 is composed of services, and the nature
|
||||
of a service is such that it exists for the lifecycle of the application.
|
||||
The class for a particular service can be re-used and registered with
|
||||
multiple names (creating multiple services with the same implementation
|
||||
but perhaps different configuration), but that's only a clean scenario when
|
||||
there is just one service. PuterFSv1, on the other hand, is like an
|
||||
imaginary service composed of other services.
|
||||
|
||||
The following possibilities then should be discussed:
|
||||
- CompositeService base class for a service that is composed of
|
||||
more than one service.
|
||||
- Refactor filesystem to not use service architecture.
|
||||
- Each filesystem service can manage state and configuration
|
||||
for multiple mountpoints
|
||||
(I don't like this idea; it feels messy. I wonder what software
|
||||
principles this violates)
|
||||
|
||||
We can take advantage of traits/interfaces here.
|
||||
PuterFSv1 depends on two interfaces:
|
||||
- An S3-like data storage implementation
|
||||
- An fsentry storage implementation
|
||||
|
||||
Counterintuitively from what I first thought, "Refactor the filesystem"
|
||||
actually looks like the best solution, and it doens't even look like it
|
||||
will be that difficult. In fact, it'll likely make the filesystem easier
|
||||
to maintain and more robust as a result.
|
||||
|
||||
Additionally, we can introduce PuterFSv2, which will introduce storing
|
||||
data in chunks identified by their hashes, and associated hashes with
|
||||
fsentries.
|
||||
|
||||
PuterFSService will be a new service which registers 'PuterFSv1' with
|
||||
FilesystemService.
|
||||
|
||||
An instance of a filesystem needs to be separate from a mountpoint.
|
||||
For example, PuterFSv1 will usually have only one instance but it may
|
||||
be mounted several different times. `/some-user` on Puter's VFS could
|
||||
be a mountpoint for `/some-user` in the instance of PuterFSv1.
|
Loading…
Reference in New Issue
Block a user