Available Datasets
Note
If you would like an additional dataset installed for general use please use the following form or contact us though the ticketing system.
Some commonly used datasets have been deployed already:
Available Datasets in Galvani
| Cluster | Dataset Name | location |
|---|---|---|
| Galvani | ImageNet-C |
/scratch_local/datasets/ImageNet-C |
| Galvani | Imagenet2012 |
/scratch_local/datasets/ImageNet2012 |
| Galvani | Imagenet-r |
/scratch_local/datasets/imagenet-r |
| Galvani | ImageNet2012_val.tar |
/mnt/qb/datasets/ImageNet2012_val.tar |
| Galvani | ImageNet-ffcv |
/mnt/qb/datasets/ImageNet-ffcv |
| Galvani | CLEVR_v1.0 |
/mnt/qb/datasets/CLEVR_v1.0 |
| Galvani | cl_ssl_ica |
/mnt/qb/datasets/cl_ssl_ica |
| Galvani | coco |
/mnt/qb/datasets/coco |
| Galvani | Falcor3D_down128 |
/mnt/qb/datasets/Falcor3D_down128 |
| Galvani | ffcv_imagenet_data |
/mnt/qb/datasets/ffcv_imagenet_data |
| Galvani | imagenet-styletransfer |
/mnt/qb/datasets/imagenet-styletransfer |
| Galvani | kitti |
/mnt/qb/datasets/kitti |
| Galvani | laion400m |
/mnt/qb/datasets/laion400m |
| Galvani | ModelNet40 |
/mnt/qb/datasets/ModelNet40 |
| Galvani | NMR_Dataset |
/mnt/qb/datasets/NMR_Dataset |
| Galvani | stl10_binary |
/mnt/qb/datasets/stl10_binary |
| Galvani | WeatherBench |
/mnt/qb/datasets/WeatherBench |
| Galvani | yfcc100m |
/mnt/qb/datasets/yfcc100m |
| Galvani | yfcc15m |
/mnt/qb/datasets/yfcc15m |
Datasets on Galvani Compute Nodes
We have also deployed some commonly used datasets locally on compute nodes on select partitions for faster I/O in your jobs. Here is a list of currently available datasets:
| Dataset | location | partition |
|---|---|---|
| Imagenet-c | /scratch_local/datasets/ImageNet-C |
2080-galvani and a100-galvani |
| Imagenet | /scratch_local/datasets/ImageNet2012 |
2080-galvani and a100-galvani |
| Imagenet-r | /scratch_local/datasets/imagenet-r |
2080-galvani and a100-galvani |
Note
On request we can manually deploy datasets on a subset of nodes, which are then selectable with SLURM features/constraints. To request this, please contact us.