SDFS XML Configuration Parameters
Tag: locations
| Attribute | Description | Editable after 1st mount |
| dedupe-db-store | The location of the map metadata. This should be located on local disk. Default is /opt/sdfs/volumes/<volume-name>/ddb | Yes. All files will need to be copied to new location |
| io-log | ignore |
<locations dedup-db-store=”C:\Program Files\sdfs\volumes\b2\ddb” io-log=”C:\Program Files\sdfs\volumes\b2\ioperf.log”/>
Tag: io
| Attribute | Description | Editable after 1st mount |
| chunk-size | The filesystem buffer size used for deduping data in KB. The default is set to 245KB. When –backup-volume is selected 40960KB. | No |
| claim-hash-schedule | The cron schedule for performing garbage collection. This is set to run every sunday at 12PM. Look at (http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html) for more detail. | Yes |
| dedupe-files | ignore | |
| hash-type | The hash type used for deduplication. This is set to VARIABLE_SIP by default. The other option is VARIABLE_MURMUR3 | No |
| log-level | If set to 0 opendedupe will be put into debug mode. Default is 1. | Yes |
| max-file-inactive | Time in seconds a file can be set inactive before it is close. 0 ignores this option. | Yes |
| max-file-write-buffers | Maximum size of the writebuffer, in MB, for a global internal IO cache before deduplication. This is set to 2048MB for backup-volume and 512MB by default. This should be set to at least 512. | Yes |
| max-open-files | The maximum number of open files allowed in the filesystem. This parameter determines the amount of RAM used by the filesystem. Each open file will use approximately 3x the max-file-write-buffers during active IO. Default for this parameter is 4096. When using backup-volume it is set to 16. | Yes |
| max-variable-segment-size | For variable block deduplication this is the MAX segment size in KB. The default is 32KB and 128KB for backup-volume | No |
| meta-file-cache | Determines the number of metadata files to cache in memory. Default is 1024 | Yes |
| min-variable-segment-size | The minimum chunk size in bytes at which deduplication will start. This is set to 4096 by default | No |
| read-ahead | For cloud storage, initiates readahead cache for files on open. Default is true. | Yes |
| safe-close | Closes files when operating system initiates close request. Set this to false when exporting through NFS. Default is true | Yes |
| safe-sync | Syncs files when operating system initiates sync request. Set this to false when exporting over NFS Default is true. | Yes |
| variable-window-size | Sets the Rabin window size in bytes for variable block deduplication. This is set to the optimal parameter. The default is 48 bytes. | No |
| volume-type | The type of volume. This option does nothing Options are standard and backup. Default is backup. | No |
| write-threads | The number of simultanious deduplication IO tasks that are performed. Default is 2x the number of cores or 8, whichever is greater. | Yes |
Tag: permissions
Description: Posix file permissions on newly created files
| Attribute | Description | Editable after 1st mount |
| default-file | The default unix file permissions. Default is 0644 | Yes |
| default-folder | The default folder permissions. Default is 0755 | Yes |
| default-group | Posix group owner for new files. Default is 0 | Yes |
| default-owner | Posix file owner. Default is 0. | Yes |
Tag: sdfscli
Description: REST API parameters
| Attribute | Description | Tunable after 1st mount |
| enabled | Enable listen port | Yes |
| enable-auth | Authentication required for control port. Default is false | Yes |
| listen-address | The host ip that the rest port listens on. Default is localhost | Yes |
| password | SHA 256 value of password | No – must be set thought sdfscli |
| salt | The salt used to configure password | No – determined when password set. |
| use-ssl | Use TLS for all REST communication. Default is true | Yes |
Tag: local-chunkstore
Description: DSE parameters
| allocation-size | The maximum size in bytes of the backend dedupe storage before compression. | Yes |
| average-chunk-size | The Average chunk size in bytes of data after deduplication. This is used to determine the number of hashtable entries to expect on a completely full volume. The default is 8192. | Yes |
| chunk-store | The location where unique blocks will be stored or cached. The default is in /opt/sdfs/volumes/<volume-name>/chunkstore/chunks | Yes. Data will need to be copied to new location |
| chunkstore-class | The class that is used to store the chunks. This is superseded by extended-config sub key chunkstore-class parameter | No |
| cluster-config | Ignore | |
| cluster-dse-password | Ignore | |
| compress | Data is compressed or not. Default is true | No |
| disable-auto-gc | Disable automatic garbage collection as data fills up. Garbage collection will happen at 10% full intervals if set to false. Default is false. | Yes |
| enabled | ignore | |
| encrypt | Deduplicated data encryption is turned on | Yes |
| encryption-iv | Default Initialization vector used for encryption. This value is static for some encryption activities. | No |
| encryption-key | The encryption key used for AES 256 bit encryption | No |
| fpp | ignore | No |
| gc-class | Class used to manage garbage collection. Do not change the default. Default is org.opendedup.sdfs.filestore.gc.PFullGC | No |
| hash-db-store | The local location of the hashtable. This should be on the fastest disk possible. Default is /opt/sdfs/volumes/<volume-name>/chunkstore/hdb-<randomid> | Yes. Data must be copied to new location |
| hash-db-class | Hash db class used for the hashtable. Default is org.opendedup.collections.RocksDBMap | No |
| low-memory | ignore | No |
| io-threads | ignore | No |
| max-repl-batch-sz | The maximum batch size a response will accept. Default is 128 | Yes |
Tag: network
Description: Deprecated network parameters. Ignore.
Tag: extended-config
Description: Extended configuration parameters for chunk storage.
Special note: This table includes sub tag parameters as well.
| allow-sync | Allow file syncs to also force local data spool to flush to cloud. Default is false | Yes |
| block-size | The data block size, in MB, used for Archive files this is set between 25MB-40MB by default | No |
| delete-unclaimed | Delete data in the cloud when it is deleted locally. Default is true | Yes |
| io-threads | Specifies the number of upload threads uses to send data to the cloud. The default is 16 but can be safely set to 64 for fast connections over 1.5GB/s | Yes |
| local-cache-size | The size of the local cache when sending data to the cloud. When using with backup on Windows, this should be set to at least the size of the largest single backup. This is set to 10GB by default | Yes |
| map-cache-size | The number of open archive file maps allowed at any time. This is set to 100 by default. | Yes |
| read-speed | The throttled read speed to cloud storage in KB/s. The default is 0 (disabled) | Yes |
| service-type | The object storage service used. This attribute is optional. This should only be set by the mkfs.sdfs command | No |
| sync-check-schedule | The schedule that is used to make sure metadata in the cloud is the same as the local metadata.Default is 4 59 23 * * ?. Take a look at http://www.quartz-scheduler.org/documentation/quartz-2.x/tutorials/crontrigger.html for more detail on how this is set | Yes |
| sync-files | Sets whether metadata is also synced to the cloud. Default is true | Yes |
| upload-thread-sleep-time | The sleep time in MilliSeconds before archive files timeout and are uploaded to the cloud regardless of size. The default is 10000 (10 Seconds) | Yes |
| write-speed | The throttled write speed to cloud or local storage. Default is set to 0 (disabled) | Yes |
| access-key | The access key used to connect to cloud storage | Yes |
| bucket-name | The bucket name used for cloud storage | No. Could be but you would want to make sure all the data is migrated. |
| chunk-store-class | The class used to manage storage. This is set based on the storage type specified during the mkfs.sdfs command. | No |
| enabled | ignore | |
| secret-key | The secret key used to authenticated to the object storage | Yes |
Tag: volume
Description : FS Service configuration parameters
| allow-external-links | Allow symlinks to external files within SDFS. Default is true | Yes |
| capacity | The logical capacity shown to the operating system. This is ignored by default but can be activated by setting use-dse-size=”false”. | Yes |
| closed-gracefully | Indicates whether the volume was shut down gracefully. Will be set to true when volume is shut down. Set to false to recover the entire hashtable from the cloud. | Yes |
| cluster-block-copies | ignore | |
| cluster-id | ignore | |
| cluster-rack-aware | ignore | |
| cluster-response-timeout | ignore | |
| compress-metadata | Indicates whether metadata map files (DDB) will be compressed using LZ4 when closed. Default is false. Set to true if –backup-volume | Yes |
| current-size | The current used capacity of the filesystem | No |
| dse-comp-size | The size of the DSE compressed | No |
| dse-size | The current size of the DSE | No |
| duplicated-bytes | The number of deduplicated bytes found | No |
| files | Number of files in the filesystem | No |
| maximum-percentage-full | The maximum percentage full the volume can get before writes will be blocked. Default is 95% | Yes |
| name | The name of the volume specified during the mkfs.sdfs command | Yes |
| offline | Determines if the volume is currently offline. This is set by internal processes. | No |
| path | The path to filesystem stubs on local disk. This is set to /opt/sdfs/volumes/<volume-name>/files by default | Yes. Files need to be copied to new location. |
| perf-mon-file | ignore | |
| read-bytes | The number of bytes read from the filesystem | No |
| read-timeout-seconds | ignore | |
| serial-number | The serial number of the filesystem determined at mkfs.sdfs. | No |
| sync-files | If set to true will initialize a fill sync of the local filesystem with data from the cloud on next mount. Set this to true to recover all files from the cloud. Default is false. | Yes |
| use-dse-capacity | Use the actual dse capacity specified by allocation-size to report maximum capacity for volume. Default is true | Yes |
| use-perf-mon | ignore | |
| volume-clustered | ignore | |
| write-bytes | Current number of bytes written | No |
| write-timeout-seconds | ignore |