Editing CCU:Global dataset storage (section)

== Writing your data to the global storage ==

The global storage is populated from the subdirectory "datasets/cluster" in your home directory on the CCU master node Lolth (ccu-master.inf.uni-konstanz.de, IP 134.34.224.84). If you try to login there with ssh, the shell is extremely limited for security reasons. However, you can run an rsync command to the server as follows. Assume the current directory contains a subdirectory "my_dataset" which you want to copy to the global storage on the cluster, then run

<syntaxhighlight lang="bash">
> rsync -avz --info=progress2 ./my_dataset your.username@ccu-master.inf.uni-konstanz.de:datasets/cluster/
</syntaxhighlight>

You now have a copy of your dataset on Lolth. What happens now is that roughly every hour, the datasets on Lolth are synced to the directory "/raid/datasets/your.username" on an NFS server.
This directory is exported and you can mount it into any container running on the cluster. Note that every user has read access to the whole directory tree, so you can use this method to share data between users as well.
As a side effect, you now also have two backups of your data on two different machines (however in the same rack, so not really fire-proof).

You can also delete data from Lolth by ssh'ing into the machine and using rm to delete stuff in the "datasets/cluster" subdirectory. During the hourly sync, data not present here will also be deleted from the global cluster storage.