Re: [tlug] Tuesday at THS: "Sync and share your data with Syncthing"?

[Kalin KOZHUHAROV <me.kalin@example.com>]

On Wed, Mar 7, 2018 at 5:04 AM, Curt Sampson <cjs@example.com> wrote:
> On 2018-03-06 14:37 +0100 (Tue), Kalin KOZHUHAROV wrote:
>
>> On Thu, Mar 1, 2018 at 2:26 PM, Curt Sampson <cjs@example.com> wrote:
>>
>> > But you can run multiple servers such that so long as any single
>> > server is running you'll be working, can't you? I'd think that setting
>> > up one or two in AWS and another one or two in Google Cloud would
>> > provide pretty darn high reliablity 'out of the box' so long as you're
>> > not making configuration changes to the servers.
>>
>> Yes, pretty much along my thoughts, though I'd use AWS and another VPS
>> provider (+backup).
>
> Right. So the biggest caveat I had with Syncthing relates to this: it
> runs in a loose mesh with trust transitive between all nodes and no
> at-rest encryption, so once one node is compromised the whole system
> is compromised. (Unlike with Dropbox or similar, it looks like you
> can't easily and centrally turn off sync for malicious actor because
> there is no central configuration.)
>
> That means that you've got a fair amount of exposure should someone
> get access to one of your cloud servers.
>
yup, this is always the case, even if you run your own
hardware/datacenter (trusting the firmware).
And is pretty standard mitigated for simple environments with
monitoring and periodic auditing.

> Possibly this could be mitigated significantly with the addition of
> at-rest encryption by using the Syncthing volume as the underlying
> storage layer for [eCryptfs] or something similar. (I'm fairly
> confident in the security of eCryptfs when used properly because it's
> what Google uses to encrypt the home dirs of users on Chromebooks.)
>
Encryption at rest in a cloud does little sense. A root compromise of
your VM gets access at the files.
For 24/7 systems there is no "at rest", it is not a laptop.

Now, eCryptfs is actual file-level encryption, so having file-level
encryption in the hosts, we shouldn't worry since only encrypted data
is being synced and the key stays at the host (so worry is pushed to
the endpoint, a scary thought).

>> >> I have been toying with the idea (offering syncthing storage) for a
>> >> few months, but not looking yet for customers.
>> >
>> > What would be the advantage here over something like Dropbox or Google
>> > Drive?....
>> >
>> Nothing much, I am prospecting new clients that does not do GSuite and
>> are unhappy or have no other cloud backup.
>> I offer "real support" since I am a one-man shop. (There are companies
>> that still prefer that)
>
> While the real support is nice, I would personally be pretty worried
> given that you'd have full access to all my files. I trust the big
> providers to be able to put appropriate restrictions in place to keep
> their staff from accessing my files and their systems are audited on a
> regular basis.
>
Yes, if I was "just another company/person". Not if I already fix your
networks, do your non-cloud backup and generally have all root/admin
credentials :-)
Since there is no "staff", (except hosting providers' staff) it is a clear cut.
I am responsible from sales to user-support, known by name and face
and if something goes south, everyone knows who to blame :-|
 (not the same if I were say 5-people company).

>> >> And I hate that there is no proper support for hard links (I want to
>> >> keep a big (5-10 GB) store of things (e.g. maps) shared read-only
>> >> between many devices, with "root" on a server; yes this is NOT P2P
>> >> mode, but I need it).
>> >
>> > How do hard links help with this?
>> >
>> specific example: I use OSMAnd+ (paid) on 4 different devices, it
>> downloads huge maps (few GB) each month. Unfortunately there are other
>> data in that directory (e.g. indexes) that is somewhat device
>> specific.
>> I want to download the updates only on one device, propagate to the
>> others and keep one copy on a server. The only workable way is to
>> duplicate the respective device directory to the server (say 3GB), for
>> each device (=12GB).
>> Then remembering to update only one of the devices, after
>> syncthing-ing to the server, semi-manually copy (i..e hardlink) the
>> maps to other devices folders, so they get the updates.
>> If hardlinks were supported (i.e. overwrite the file in place as
>> opposed to rm&create), updating the maps on one device will magically
>> (via the hard link) update the rest.
>
> Ah, I see. So the issue is that when a new version of the file is
> uploaded it gets a new inode (presumably so the backup system, if
> enabled, can keep the old file). What happens if you use a symlink? Is
> it followed, or sync'd as a symlink? (I guess the latter would be the
> behaviour we normally want.)
>
never followed, per design (last time I checked their issues).

Update-in-place has been discussed, that may help me:
https://github.com/syncthing/syncthing/wiki/In-Place-Updates

> Still, a relatively trivial sync script running in the background
> could deal with this for you. Though honestly, when you're talking
> about saving only a few GB or 10s of GB of space on a server these
> days, it's not something I'd be particularly worried about.
>
well, I also am thinking to run my "forensic storage" (hundreds of
drive images, 1GB-3TB each) eventually on syncthing :-D
Also, since most of those images are sparse, using eCryptfs is out of
the question there (it has no support, os it will blow up the stored
size) :-|

Cheers,
Kalin.