mirror of
https://github.com/HeyPuter/puter
synced 2024-11-14 14:03:42 +00:00
doc: add brainstorming for mountpoints
This commit is contained in:
parent
7980aafbd9
commit
9535100c85
103
doc/devlog.md
Normal file
103
doc/devlog.md
Normal file
@ -0,0 +1,103 @@
|
|||||||
|
## 2024-10-16
|
||||||
|
|
||||||
|
### Considerations for Mountpoints Feature
|
||||||
|
|
||||||
|
- `_storage_upload` takes paramter `uuid` instead of `path`
|
||||||
|
- S3 bucket strategy needs the UUID
|
||||||
|
- If we do hashes, 10MB chunks should be fine
|
||||||
|
- we're already able to smooth out bursty traffic using the
|
||||||
|
EWA algorithm
|
||||||
|
- Use of `systemFSEntryService`
|
||||||
|
- Is that normalized? Does everything go through this interface?
|
||||||
|
- Storage interface has methods like `post_insert`
|
||||||
|
- as far as I can tell this doesn't pose any issue
|
||||||
|
-
|
||||||
|
|
||||||
|
### Brainstorming Migration Strategies
|
||||||
|
|
||||||
|
#### Interface boundary at HL<->LL filesystem methods
|
||||||
|
|
||||||
|
-- **tags:** brainstorming
|
||||||
|
|
||||||
|
From the perspectice of a trait-oriented implementation,
|
||||||
|
which is not how LL/HL filesystem operations are currently implemented,
|
||||||
|
the LL-class operations are implemented in separate traits.
|
||||||
|
|
||||||
|
The composite trait containing all of these traits would be the trait
|
||||||
|
that represents a filesystem implementation itself.
|
||||||
|
|
||||||
|
Other filesystem interfaces that I've seen, such as FUSE and 9p,
|
||||||
|
all usually have a monolithic interface - that is to say, an interface
|
||||||
|
which includes all of the filesystem operations, rather than several
|
||||||
|
interfaces each implementing a single filesystem operaiton.
|
||||||
|
|
||||||
|
Something about the fact that the LL-class operations are in separate
|
||||||
|
classes makes it difficult to reason about how to move.
|
||||||
|
Is it simply that multiple files in a directory is just more
|
||||||
|
annoying to think about? Maybe, but there must be something more.
|
||||||
|
|
||||||
|
Perhaps it's that there are several references. Each implementation
|
||||||
|
(that is, implemenation of a single filesystem operation) could have
|
||||||
|
any number of different references across any number of different files.
|
||||||
|
This would not be the case with a monolithic interface.
|
||||||
|
|
||||||
|
I think the best of both worlds would be to have an interface representing
|
||||||
|
the entire filesystem and, in one place, link of of the individual
|
||||||
|
operation implementations to compose a filesystem implementation
|
||||||
|
|
||||||
|
### Filesystem Brainstorming
|
||||||
|
|
||||||
|
Puter's backend uses a service architecture. Each service is an instance
|
||||||
|
of a class extending "Service". A service can listen to events of the
|
||||||
|
backend's lifecycle, interact with other services, and interact with
|
||||||
|
external interfaces such as APIs and databases.
|
||||||
|
|
||||||
|
Puter's current filesystem, let's call it PuterFSv1, exists as the result
|
||||||
|
of multiple services working together. We have LocalDiskStorageService
|
||||||
|
which mimics an S3 bucket on a local system, and we have
|
||||||
|
DatabaseFSEntryService which manages information about files, directories,
|
||||||
|
and their relationships within the database, and therefore depends on
|
||||||
|
DatabaseAccessService.
|
||||||
|
|
||||||
|
It is now time to introduce a MountpointService. This will allow another
|
||||||
|
service or a user's configuration to assign an instance of a filesystem
|
||||||
|
implementation (such as PuterFSv1) to a specific path.
|
||||||
|
|
||||||
|
The trouble here is that PuterFSv1 is composed of services, and the nature
|
||||||
|
of a service is such that it exists for the lifecycle of the application.
|
||||||
|
The class for a particular service can be re-used and registered with
|
||||||
|
multiple names (creating multiple services with the same implementation
|
||||||
|
but perhaps different configuration), but that's only a clean scenario when
|
||||||
|
there is just one service. PuterFSv1, on the other hand, is like an
|
||||||
|
imaginary service composed of other services.
|
||||||
|
|
||||||
|
The following possibilities then should be discussed:
|
||||||
|
- CompositeService base class for a service that is composed of
|
||||||
|
more than one service.
|
||||||
|
- Refactor filesystem to not use service architecture.
|
||||||
|
- Each filesystem service can manage state and configuration
|
||||||
|
for multiple mountpoints
|
||||||
|
(I don't like this idea; it feels messy. I wonder what software
|
||||||
|
principles this violates)
|
||||||
|
|
||||||
|
We can take advantage of traits/interfaces here.
|
||||||
|
PuterFSv1 depends on two interfaces:
|
||||||
|
- An S3-like data storage implementation
|
||||||
|
- An fsentry storage implementation
|
||||||
|
|
||||||
|
Counterintuitively from what I first thought, "Refactor the filesystem"
|
||||||
|
actually looks like the best solution, and it doens't even look like it
|
||||||
|
will be that difficult. In fact, it'll likely make the filesystem easier
|
||||||
|
to maintain and more robust as a result.
|
||||||
|
|
||||||
|
Additionally, we can introduce PuterFSv2, which will introduce storing
|
||||||
|
data in chunks identified by their hashes, and associated hashes with
|
||||||
|
fsentries.
|
||||||
|
|
||||||
|
PuterFSService will be a new service which registers 'PuterFSv1' with
|
||||||
|
FilesystemService.
|
||||||
|
|
||||||
|
An instance of a filesystem needs to be separate from a mountpoint.
|
||||||
|
For example, PuterFSv1 will usually have only one instance but it may
|
||||||
|
be mounted several different times. `/some-user` on Puter's VFS could
|
||||||
|
be a mountpoint for `/some-user` in the instance of PuterFSv1.
|
Loading…
Reference in New Issue
Block a user