SDFS XML Configuration Parameters

SDFS XML Configuration Parameters


Tag: locations


Attribute Description Editable after 1st mount
dedupe-db-store The location of the map metadata. This should be located on local disk. Default is /opt/sdfs/volumes/<volume-name>/ddb Yes. All files will need to be copied to new location
io-log ignore


<locations dedup-db-store=”C:\Program Files\sdfs\volumes\b2\ddb” io-log=”C:\Program Files\sdfs\volumes\b2\ioperf.log”/>


Tag: io


Attribute Description Editable after 1st mount
chunk-size The filesystem buffer size used for deduping data in KB. The default is set to 245KB. When –backup-volume is selected 40960KB. No
claim-hash-schedule The cron schedule for performing garbage collection. This is set to run every sunday at 12PM. Look at ( for more detail. Yes
dedupe-files ignore
hash-type The hash type used for deduplication. This is set to VARIABLE_SIP by default. The other option is VARIABLE_MURMUR3 No
log-level If set to 0 opendedupe will be put into debug mode. Default is 1. Yes
max-file-inactive Time in seconds a file can be set inactive before it is close. 0 ignores this option. Yes
max-file-write-buffers Maximum size of the writebuffer, in MB, for a global internal IO cache before deduplication. This is set to 2048MB for backup-volume and 512MB by default. This should be set to at least  512. Yes
max-open-files The maximum number of open files allowed in the filesystem. This parameter determines the amount of RAM used by the filesystem. Each open file will use approximately 3x the max-file-write-buffers during active IO. Default for this parameter is 4096. When using backup-volume it is set to 16. Yes
max-variable-segment-size For variable block deduplication this is the MAX segment size in KB. The default is 32KB and 128KB for backup-volume No
meta-file-cache Determines the number of metadata files to cache in memory. Default is 1024 Yes
min-variable-segment-size The minimum chunk size in bytes at which deduplication will start. This is set to 4096 by default No
read-ahead For cloud storage, initiates readahead cache for files on open. Default is true. Yes
safe-close Closes files when operating system initiates close request. Set this to false when exporting through NFS. Default is true Yes
safe-sync Syncs files when operating system initiates sync request. Set this to false when exporting over NFS  Default is true. Yes
variable-window-size Sets the Rabin window size in bytes for variable block deduplication. This is set to the optimal parameter. The default is 48 bytes. No
volume-type The type of volume. This option does nothing Options are standard and backup. Default is backup. No
write-threads The number of simultanious deduplication IO tasks that are performed. Default is 2x the number of cores or 8, whichever is greater. Yes


Tag: permissions

Description: Posix file permissions on newly created files


Attribute Description Editable after 1st mount
default-file The default unix file permissions. Default is 0644 Yes
default-folder The default folder permissions. Default is 0755 Yes
default-group Posix group owner for new files. Default is 0 Yes
default-owner Posix file owner. Default is 0. Yes


Tag: sdfscli

Description: REST API parameters


Attribute Description Tunable after 1st mount
enabled Enable listen port Yes
enable-auth Authentication required for control port. Default is false Yes
listen-address The host ip that the rest port listens on. Default is localhost Yes
password SHA 256 value of password No – must be set thought sdfscli
salt The salt used to configure password No – determined when password set.
use-ssl Use TLS for all REST communication. Default is true Yes


Tag: local-chunkstore

Description:  DSE parameters


allocation-size The maximum size in bytes of the backend dedupe storage before compression. Yes
average-chunk-size The Average chunk size in bytes of data after deduplication. This is used to determine the number of hashtable entries to expect on a completely full volume. The default is 8192. Yes
chunk-store The location where unique blocks will be stored or cached. The default is in /opt/sdfs/volumes/<volume-name>/chunkstore/chunks Yes. Data will need to be copied to new location
chunkstore-class The class that is used to store the chunks. This is superseded by extended-config sub key chunkstore-class parameter No
cluster-config Ignore
cluster-dse-password Ignore
compress Data is compressed or not. Default is true No
disable-auto-gc Disable automatic garbage collection as data fills up. Garbage collection will happen at 10% full intervals if set to false. Default is false. Yes
enabled ignore
encrypt Deduplicated data encryption is turned on Yes
encryption-iv Default Initialization vector used for encryption. This value is static for some encryption activities. No
encryption-key The encryption key used for AES 256 bit encryption No
fpp ignore No
gc-class Class used to manage garbage collection. Do not change the default. Default is org.opendedup.sdfs.filestore.gc.PFullGC No
hash-db-store The local location of the hashtable. This should be on the fastest disk possible. Default is /opt/sdfs/volumes/<volume-name>/chunkstore/hdb-<randomid> Yes. Data must be copied to new location
hash-db-class Hash db class used for the hashtable. Default is org.opendedup.collections.RocksDBMap No
low-memory ignore No
io-threads ignore No
max-repl-batch-sz The maximum batch size a response will accept. Default is 128 Yes


Tag: network

Description:  Deprecated network parameters. Ignore.


Tag: extended-config

Description:  Extended configuration parameters for chunk storage.

Special note: This table includes sub tag parameters as well.


allow-sync Allow file syncs to also force local data spool to flush to cloud. Default is false Yes
block-size The data block size, in MB, used for Archive files this is set between 25MB-40MB by default No
delete-unclaimed Delete data in the cloud when it is deleted locally. Default is true Yes
io-threads Specifies the number of upload threads uses to send data to the cloud. The default is 16 but can be safely set to 64 for fast connections over 1.5GB/s Yes
local-cache-size The size of the local cache when sending data to the cloud. When using with backup on Windows, this should be set to at least the size of the largest single backup. This is set to 10GB by default Yes
map-cache-size The number of open archive file maps allowed at any time. This is set to 100 by default. Yes
read-speed The throttled read speed to cloud storage in KB/s. The default is 0 (disabled) Yes
service-type The object storage service used. This attribute is optional. This should only be set by the mkfs.sdfs command No
sync-check-schedule The schedule that is used to make sure metadata in the cloud is the same as the local metadata.Default is 4 59 23 * * ?. Take a look at for more detail on how this is set Yes
sync-files Sets whether metadata is also synced to the cloud. Default is true Yes
upload-thread-sleep-time The sleep time in MilliSeconds before archive files timeout and are uploaded to the cloud regardless of size. The default is 10000 (10 Seconds) Yes
write-speed The throttled write speed to cloud or local storage. Default is set to 0 (disabled) Yes
access-key The access key used to connect to cloud storage Yes
bucket-name The bucket name used for cloud storage No. Could be but you would want to make sure all the data is migrated.
chunk-store-class The class used to manage storage. This is set based on the storage type specified during the mkfs.sdfs command. No
enabled ignore
secret-key The secret key used to authenticated to the object storage Yes


Tag: volume

Description : FS Service configuration parameters


allow-external-links Allow symlinks to external files within SDFS. Default is true Yes
capacity The logical capacity shown to the operating system. This is ignored by default but can be activated by setting use-dse-size=”false”. Yes
closed-gracefully Indicates whether the volume was shut down gracefully. Will be set to true when volume is shut down. Set to false to recover the entire hashtable from the cloud. Yes
cluster-block-copies ignore
cluster-id ignore
cluster-rack-aware ignore
cluster-response-timeout ignore
compress-metadata Indicates whether metadata map files (DDB) will be compressed using LZ4 when closed. Default is false. Set to true if –backup-volume Yes
current-size The current used capacity of the filesystem No
dse-comp-size The size of the DSE compressed No
dse-size The current size of the DSE No
duplicated-bytes The number of deduplicated bytes found No
files Number of files in the filesystem No
maximum-percentage-full The maximum percentage full the volume can get before writes will be blocked. Default is 95% Yes
name The name of the volume specified during the mkfs.sdfs command Yes
offline Determines if the volume is currently offline. This is set by internal processes. No
path The path to filesystem stubs on local disk. This is set to /opt/sdfs/volumes/<volume-name>/files by default Yes. Files need to be copied to new location.
perf-mon-file ignore
read-bytes The number of bytes read from the filesystem No
read-timeout-seconds ignore
serial-number The serial number of the filesystem determined at mkfs.sdfs. No
sync-files If set to true will initialize a fill sync of the local filesystem with data from the cloud on next mount. Set this to true to recover all files from the cloud. Default is false. Yes
use-dse-capacity Use the actual dse capacity specified by allocation-size to report maximum capacity for volume. Default is true Yes
use-perf-mon ignore
volume-clustered ignore
write-bytes Current number of bytes written No
write-timeout-seconds ignore