Let's talk about a Mini-DOS

This is the best place to discuss about the various Oric operating systems like Sedoric, Randos, FT-Dos, and others, as well as serious software, utilities, word processors, disassemblers, etc... that runs on oric computers.
User avatar
Dbug
Site Admin
Posts: 4437
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Let's talk about a Mini-DOS

Post by Dbug »

In these last years, I've been talking with a number of people that had (like me) some large scale projects on the Oric.

Some wanted to port games, some wanted to port applications, some wanted to create demos.
The common point of all these projects is that they use the floppy disc drive, the other common point is that the Oric operating systems are not adapted to these projects.
The reason is simple. The Oric DOSes are mainly way to add disc support for BASIC users. I will take the example of Sedoric.

I you look at the Sedoric manual, you will see that it implements the following new BASIC instructions:

Directory: DIR,LDIR
Load and Save: LOAD,SEARCH,SAVE,SAVEO,SAVEM,SAVEU,ESAVE
Erasing: DEL,DESTROY,DELBAK
File changes: REN,STATUS,PROT,UNPROT
Sector access: PMAP,SMAP,CRESEC,FRSEC
Disc operations: INIT,BACKUP,COPY,DSYS,DNAME,INIST,DKEY,TRACK,DTRACK,DNUM,SYSTEM
Programing commands: RENUM,MERGE,DELETE,SEEK,CHANGE..TO..,NUM,NUM END
Keyboard handling: KEY,KEYIF,QWERTY,AZERTY,ACCENT,KEYDEF,KEYUSE,KEYSAVE,VUSER
String handling: TKEN,UNTKEN,STRUN,INSTR
Error handling: ERR SET,ERR OFF,ERRGOTO,RESUME,ERROR
Formated I/O: LINPUT,CREATEW,WINDOW,USING,LUSING,WIDTH
Printer: OUT,PR SET,PR OFF
Graphics: LINE,BOX,LCUR,HCUR
Misc: USER,],QUIT,RESET,RESTORE,MOVE,OLD,RANDOM,SWAP
Files: OPEN,CLOSE,PUT,TAKE,APPEND,REWIND,JUMP,BUILD,TYPE,LTYPE,&(),PUT,TAKE,FIELD,LSET,RSET

I guess that everybody will agree on the fact that most of this is of absolutely no use for people that are not using BASIC as their programming language of choice, specially considering there is no easy way to call any of these functions from an assembly code routine.

And the fact that the notion of handle, partial file loads, folders structures, etc are missing... makes it impossible to implement something like the C standard library.

Sedoric is quite good as a DOS. You can list files, launch commands, copy stuff between discs, etc...

But quite frankly, if you write games or demos, you have very specific needs:
- The maximum amount of free room both on disc and in memory
- Easy and fast way to load data to memory (not necessarily stored as files with filenames and dates, just bunches of data)
- Eventually possibility to save data (like high score tables for a game)
- Being compatible with a maximum of hardware (oric 1/atmos/telestrat but also microdisc/jasmin)

The Defence Force demos actually only needed a simple ReadSector routine associated with a small table containing three informations:
- track/sector position of the first sector of the file to load
- number of sectors to read
- adress in memory of where to put data

Obviously this is a primitive directory structure, but since all the data were generated with a floppy maker tool, it was possible to get these information as a generated source code included in the loader to avoid having to load the table at startup.

A real new os good enough to be able to get something like Contiki to run, would require at least the following:

- Hardware level: Disc structure informations, ReadSector/WriteSector
- File level: Directory structure and sector allocation management
- Library level: Buffered loads, file handling, etc...

I am not sure that beeing compatible with Sedoric at structural level is a good idea, because the disc layout is quite complicated. If necessary, Sedoric format compatibility could be implemented easily as a small C conversion program that would use the three layers describes previously.

What would be an ideal format ?

Obviously, in a form or another we need to get for each file:
- name
- size in bytes
- location on disc

If we assume (like in a demo) that we do not have fragmented files, then the location can just be an indication as track/sector of the first sector.
But if we use if in normal OS usages patterns, we will need to create and erase files, so this will lead to some fragmentation. (of course it is still possible to defragment in realtime when writing new files, but this will kill the floppies very fast, so this is not a good idea).

So if we handle fragmentation, this means that we need the location of each single sector of the file.

The size taken by of all these data depends of three parameters:
- The maximum number of files we can store on the disc
- The number of sectors that are allocated on the disc
- The size of individual metadata for each file (lenght of filename for example).

If we have a (fat) disc that contains 2 sides, 82 tracks per side, and 18 sectors (of 256 bytes), we have 2x82x18=2952 sectors that can be possibly allocated. A candid approach of the problem is to store positions as simple sectors numbers in the range 0-2951, and using a 12 bits value for that purpose.

Many other operating systems have the notion of cluster. Instead of allocating sectors one by one, you allocate them by blocks of fixed size. This diminished the size of metadata to keep, but also increase the amount of waste when storing size.

With a cluster of size=2 sectors, we now have only 1476 clusters to manage, but it also means that each file is rounded to the upper 512 bytes limit. With a cluster size of 4 sector, we have only 738 cluster, with files rounded to 1024 bytes.

All suggestions about ideal disc layout are welcome !

You can read more about some existing floppy formats here:
Wikipedia
MS-Dos/AtariST format
User avatar
Euphoric
Game master
Posts: 99
Joined: Mon Jan 09, 2006 11:33 am
Location: France

Post by Euphoric »

Obviously, building a new (even mini-) OS is a big task. So I would suggest it should not only ease the task of developing demoes/games/applications, but also improve the transfers between the Oric and the other main platforms.

This said, there's not much choice. Even if the MSDOS format is not the best, I would say the filesystem *must* be MSDOS FAT12. Then, if we intend to progress on projects such as the IDE interface or the memory cards interface, we should have at least MSDOS FAT16. Considering that those harddisks and memory cards are always bigger than 32 MBytes today, this means that we should also support clusters. Hopefully we should be able to avoid FAT32, since FAT16 is so wide spread.

The advantage of using a MSDOS format is that we wouldn't need to bring a full DOS from the beginning... We could progressively bundle features, e.g start with read routines, and not cope with directories at first (but this will be necessary when/if bigger storage comes).

However, this is still a big project, and if you aren't ready to develop and/or wait for it, you could have a go at Randos... Randos is not as featured-full as Sedoric when you use Basic, but it has several features not found in Sedoric that makes it more modern and C-compatible...

Randos has buffered files that you can read byte by byte, making possible to write un-archivers for example (byte access is not much optimized, so it's a bit slow, but it works). Randos has 512-bytes sectors instead of 256-bytes sectors, so you can read/write a non-system Randos disk on a PC much more easily than a Sedoric disk.
Randos has a 12-bit FAT, though it is not MSDOS.
Randos has directories...

All these reasons made me choose it as the basis for a hard-disk operating system. This Randos 2.0 runs today on Euphoric with an emulation of the IDE interface, even if I have not been able to test it on real hardware due to electrical/timing problems with the IDE interface...

Cheers,

Fabrice
User avatar
Dbug
Site Admin
Posts: 4437
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Post by Dbug »

Euphoric wrote:Obviously, building a new (even mini-) OS is a big task. So I would suggest it should not only ease the task of developing demoes/games/applications, but also improve the transfers between the Oric and the other main platforms.
I agree that being able to read on the oric something written on a PC, Amiga, Atari, etc... would be very nice and practical.
Euphoric wrote:This said, there's not much choice. Even if the MSDOS format is not the best, I would say the filesystem *must* be MSDOS FAT12.
Well, in my idea it was not that is *has* to be MSDOS compatible, more that it has to be *able* to read and write MSDOS discs.
That makes a lot of differences.
In the same way, this system has to be able to read and write discs using other Oric DOS formats (Sedoric, XL-Dos, Randos, FT-Dos, ...).

Anyway, to achieve such a thing, the functionalities does not have to be built in into the operating system. A perfect example is Amiga OS. It has a built-in system that accepts 'plugins' that can cope with non native filesystems.

My idea was to get a minimalistic kernel that can be used to bring basic read/write functionalities to games and demos, compatible with all known disc drives, while leaving a maximum amount of memory available.

Things like Sedoric & MSDOS file system compatibility can be additional layer developed by other people on top of the hardware abstraction layer.
Euphoric wrote:Then, if we intend to progress on projects such as the IDE interface or the memory cards interface, we should have at least MSDOS FAT16. Considering that those harddisks and memory cards are always bigger than 32 MBytes today, this means that we should also support clusters. Hopefully we should be able to avoid FAT32, since FAT16 is so wide spread.
From a program point of view, reading files, writing files, accessing directory information, etc... should be independant of the underlying file system.

From a hardware point of view, talking to an IDE interface, accessing a microdisc, accessing a jasmin, accessing a flash card, ... are all different operations, but that can logicaly have a common 'interface' to hide the different implementations:
- getting device information (avilable/connected/changed, capacity, number of tracks/sectors/plates/whatever)
- format/prepare the device so it can be usable
- write a bloc of data
- read a bloc of data
- anything else I forgot...

I guess we can consider each of these small hardware implementation as 'drivers' that need to be loaded (mounted) before the device can be used.

Idealy if we can figure out a standardized interface, and a standardized format/method of loading, anyone could implement new devices and get them to work with old programs out of the box.
Euphoric wrote:The advantage of using a MSDOS format is that we wouldn't need to bring a full DOS from the beginning... We could progressively bundle features, e.g start with read routines, and not cope with directories at first (but this will be necessary when/if bigger storage comes).
In the same way we can abstract hardware devices with a common interface, we can probably abstract the filesystems by providing a standard interface to the way to handle files and directory structure.
Euphoric wrote:However, this is still a big project, and if you aren't ready to develop and/or wait for it, you could have a go at Randos... Randos is not as featured-full as Sedoric when you use Basic, but it has several features not found in Sedoric that makes it more modern and C-compatible...

Randos has buffered files that you can read byte by byte, making possible to write un-archivers for example (byte access is not much optimized, so it's a bit slow, but it works).
Ok, do you have documentation about Randos somewhere ?
Euphoric wrote:Randos has 512-bytes sectors instead of 256-bytes sectors, so you can read/write a non-system Randos disk on a PC much more easily than a Sedoric disk.
How does this work ?
Is the microdisc eprom able to cope with both 256 bytes and 512 bytes sectors formated floppies ?
How does it detect the format ?

Anyway, if 512 bytes sectors are good for system compatibility, for efficient way of reading data from the floppy disc on a 6502 processor I'm not sure it's very efficient.
Euphoric wrote: Randos has a 12-bit FAT, though it is not MSDOS.
Randos has directories...

All these reasons made me choose it as the basis for a hard-disk operating system. This Randos 2.0 runs today on Euphoric with an emulation of the IDE interface, even if I have not been able to test it on real hardware due to electrical/timing problems with the IDE interface...
Is Randos 2.0 a patched version you made ?
User avatar
Euphoric
Game master
Posts: 99
Joined: Mon Jan 09, 2006 11:33 am
Location: France

Post by Euphoric »

Well, in my idea it was not that is *has* to be MSDOS compatible, more that it has to be *able* to read and write MSDOS discs.
That makes a lot of differences.
In the same way, this system has to be able to read and write discs using other Oric DOS formats (Sedoric, XL-Dos, Randos, FT-Dos, ...).
I agree that having a modular operating system that has modules able to read/write many OS formats would be great, but would require a lot of work... IMHO, it would be much easier and practical to give up the old Oric DOS formats and implement a single file system (MSDOS FAT), this way the disks will be readable on a PC/Mac/Atari/Amiga/... and you will be able to use the modern platforms to transfer files to the Oric...
do you have documentation about Randos somewhere ?
There's a HELP system on Randos system disk, and for french readers a translation has been put on http://oric.free.fr/DISKS/randos.doc

Euphoric wrote:
Randos has 512-bytes sectors instead of 256-bytes sectors, so you can read/write a non-system Randos disk on a PC much more easily than a Sedoric disk.

How does this work ?
Is the microdisc eprom able to cope with both 256 bytes and 512 bytes sectors formated floppies ?
How does it detect the format ?
The FDC controller can cope with 128, 256, 512 and 1024-bytes sectors. Then it's a matter of programming... The eproms of the different controllers have different routines for the reading of the boot sector, and they don't detect the size of the sectors: as a result, some of them store the second half of a 256-bytes sector in the same page as the first half, thus overwriting the first part... The Fantasmagoric boot sector I have posted in the other thread takes care of this. Anyway, once the operating system is loaded, the eprom routines aren't used anymore: so it's up to your operating system to handle 512-bytes sectors...

Anyway, if 512 bytes sectors are good for system compatibility, for efficient way of reading data from the floppy disc on a 6502 processor I'm not sure it's very efficient.
The sector read/write routines of OricDos/Sedoric/Stratsed already are 512-bytes-sector compatible...
Loading a single 512-bytes sector is faster than loading two 256-bytes sectors, and you can safely store nine 512-bytes sectors on a double density disk track, whilst storing eighteen 256-bytes sectors with Stratsed has always brought problems...
Is Randos 2.0 a patched version you made ?
Sure... :-)
[adv. mode ON]
It adds support for special "partitions" on the hard-disk, that are viewed like additionnal floppy drives (E, F, G, H, I, ...). IDE routines have been inserted, and some commands have been slightly patched to support the larger number of "drives", and to support format/backup/restore of those special partitions that can viewed as virtual floppies... Also, the boot floppy uses a uniform 512-sectors format with a special boot sector comparable to the Fantasmagoric boot sector (the original Randos system disk has an hybrid format: a small OricDos file system with 256-bytes sectors to store the system, and then a Randos file system with 512-bytes sectors for the Randos files).
Finally, when used with an also patched microdisc eprom, Randos 2.0 can be loaded from the harddisk, without need for a boot floppy.
[adv. mode OFF]

Cheers,

Fabrice
User avatar
Twilighte
Game master
Posts: 819
Joined: Sat Jan 07, 2006 12:07 am
Location: Luton, UK
Contact:

Post by Twilighte »

Then, if we intend to progress on projects such as the IDE interface or the memory cards interface, we should have at least MSDOS FAT16.
Considering that those harddisks and memory cards are always bigger than 32 MBytes today, this means that we should also support clusters.
I still fail to see the point of supporting a hard disk or flash card since both devices are currently unavailable and will never be widespread.
Also, the further we delve into extra things to handle in a Disc management systems, the bigger the footprint will become, which moves away from the original mini-dos idea.
User avatar
Dbug
Site Admin
Posts: 4437
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Post by Dbug »

I guess we identified two main needs:

1) A more evolved operating system that would be closely compatible to something that exist on other machines in order to ease the communication between Oric and other machines (ex: MS-DOS compatibility to make it easy to read and write from a PC), while also having enough flexibility to handle things like evolved storage devices (hard drives and memory cards).

2) A mini dos, that is just good enough for abstracting hardware differences between the various oric disc systems, allowing to easily makes games and demos by letting a maximum amount of free memory for the user.

I guess the first one is more adapted for things that are handling with a lot of random disc access, file being created, etc... while the second one is better suited for things where a static layout of data is good enough.
User avatar
ibisum
Wing Commander
Posts: 1643
Joined: Fri Apr 03, 2009 8:56 am
Location: Vienna, Austria
Contact:

Re: Let's talk about a Mini-DOS

Post by ibisum »

What I really want, VIM.COM (editor), CC.COM (c compiler), on a floppy whereby I can, in C on an existing Oric/Atmos/Telestrat, do virtmem on the disk files, plus coroutines. And then a micro-C-like cmd/editor/exec line interface that can at least, in C, write/read files, save/reload apps from disk dynamically, would be the next step.

In any case, a good text editor for the Oric shouldn't be that difficult, and then the ability to parse the file, well .. surely the Oric can do that onboard. Were there ever tools that could do such things, one wonders ..
Post Reply