Skip to content

Handle your data

Storage#

Several volumes of data storage are available on the IFB Core Cluster. Use it knowingly.

Usage Quota (default) Backup policy
/shared/home Home directory (personal data) 100GB False
/shared/projects Scientific and project (common data) 250GB False
/shared/bank Read-Only. Common banks (UniProt, RefSeq, ...)

If you need or plan to use more storage space, please contact us: community support website

Backup#

⚠️ There is no backup for the main storage.

Snapshots#

Some snapshots are available to protect against deletion by error but only one by day and for 5 days.

Project#

A project is a directory on the storage associated with users who can access it.

In order to have your folder and space on /shared/projects/<myproject\>, you have to request a new project on the IFB Core Cluster Account Manager: https://my.cluster.france-bioinformatique.fr/manager2/project

You can have multiple projects (and it's recommended).

More information on The project concept at IFB Core

Transfer#

SSH protocol (or SFTP - SSH File Transfer Protocol) is the only protocol available. But, you can also use clients on the cluster to download your data (wget, curl, scp, rsync, wget, ftp, etc.).

You can get/put your data as described below.

Command line#

On your workstation or on the cluster, you can use command line with your favorite client: scp, rsync, etc.

SFTP Graphic client#

Use a graphic client like FileZilla on your workstation.

Or use your file manager directly if it's possible (SFTP support).

SSHFS Client#

Use your file manager with SSHFS

Editors#

Several editors and tools are available on the cluster:

  • vim
  • emacs
  • nano

Or graphical editors. Graphic tools need some configuration: see Export display.

  • xemacs
  • geany
  • gedit

Or use web interface:

Export display#

Linux / Mac#

Simply use -Y option:

ssh -Y <username>@core.cluster.france-bioinformatique.fr

Windows#

1- Install a X server (for example: VcXsrv Windows X Server)
2- Launch the X server
3- Activate X11 forwarding

> Example with PuTTY
X11 Forwarding with PuTTY

Once connected, you can launch your program with a graphical user interface.

Manage#

Several possibilities to browse, manage and edit or view your data/scripts

  • Directly on the cluster with the command line

  • Get the data back on your workstation (see Transfer) and work locally.

  • Use SSHFS on your workstation (see above: SSHFS)

Analysis of disk space used#

Status_bars#

When you connect to ssh, you should have a view of your project spaces and their quotas (size, number of files) with the status_bars.

project1 [##------------------]      36 /     250 GB
project2 [#################---]     390 /     450 GB
Update: 2024-07-31 14:00 - Your current default account is admin - More info: status_bars --help

⚠️ status_bar is updated hourly. So it may still display the old quota though you've freed up space.

Disk usage analyzer: NCurses Disk Usage ncdu#

ncdu (NCurses Disk Usage) is a command-line tool that lets you quickly and efficiently analyse disk space usage on your system. It lists the size of each folder in a simplified interface. It is based on the du command but is much more user-friendly.

cd /shared/project/project1/
ncdu

Some major keys (press the key when ncdu is opened):

  • c Display the number of items/files
  • n Sort by name (ascending/descending)
  • s Sort by size (ascending/descending)
  • C Sort by items (ascending/descending)
  • M Sort by mtime (-e flag)
  • ? Open help screen

Some complement documentations: https://dev.yorhel.nl/ncdu