i have binary logical data, want save file in least amount of space possible. when check data size matlab workspace shows, 103 kb when save using fwrite ubit1 expands 105 kb? can save in least possible space?
there no overhead (or may meant metadata) added function fwrite
in matlab). function "low level" comes , on given machine give similar results equivalent low level functions in c
, c++
, many more languages.
to access disk, rely on lower level functions, driven filesystem
of disk , operating system. between different disks, filesystems , os may observe small differences in final result, on given system (disk/fs/os), matlab fwrite
similar every other language, , there no "overhead".
now size of data versus size of file versus size on disk:
consider following snippet:
nbits = 376 ; = true( nbits , 1 ) ; fid = fopen( 'testsize.bin' , 'w' ) ; fwrite( fid , , 'ubit1' ) ; fclose(fid) ;
this create array of 376 logical, write them format ubit1
onto disk.
before @ file, notice that, mentioned in horchler comment, in memory matlab still uses full byte (8 bits) each logical (boolean).
>> whos name size bytes class attributes 376x1 376 logical
this not problem however, since when fwrite
write on disk, format ubit1
tell use (single) significant bit horchler commented, file 1/8th of size of variable in memory...
or ??
if @ file explorer, ouch:
(this done on pc, windows 8, ntfs file system.)
1kb, naaaah, because thing not designed display sizes smaller that, it's rounded.(unix/linux user may better display hey i'm on windows have deal it).
to better information, have query more details, once access properties of file, get:
pfeeew. 47 bytes. sounds right. let's see 376/8=47
, yep that's perfect!
note "size on disk" of whopping 4kb. why need space store poor 47 bytes ? has "default allocation" size of filesystem on disk, , it's 1 of these things fwrite
cannot example. managed os/file system.
now if lot of disk being wasted, still managed information, file 47 bytes. success? ... not yet.
i choose 376 bits @ beginning @ random, because perfect multiple of 8. let's try run same code above, except we'll start with:
nbits = 377 ;
the code runs fine. file still appears 1kb in explorer know it's false, property shows:
377/8 = 47.125
, not 48, "rounded" explorer again. no!
the file size 48 byte (not 1 bit less or more). (but useful information inside file occupy 47 byte , 1 bit, last 7 bits undetermined (or pegged '0' may don't sure).
what happened behind scene fwrite
aggregating bits write group of 8, building full byte, writing full byte on disk (or bigger groups even). behind scene, has because filesystem (yes him again) not let him address individual bit on disk. filesystem expect packets of @ least byte (or more). when reaching last single bit write, fwrite
had pad 7 other bits before telling file system write on disk.
i not expert on flavours of filesystem, doubt many allow address single bit, minimum rounding should expect @ least byte ... if not more.
summary
fwrite
not introduce overhead, or 1 forced hardware , filesystem (in case other function not better).