Mac68k Forums

Home


Welcome, Guest
Guest Settings
Help

Mac68k Forums » Development » Software Hacking

Thread: Serial Disk Driver


Reply to this Thread Reply to this Thread Search Forum Search Forum Back to Thread List Back to Thread List

Permlink Replies: 72 - Pages: 5 [ Previous | 1 2 3 4 5 | Next ] - Last Post: Oct 29, 2013 12:19 PM Last Post By: bbraun Threads: [ Previous | Next ]
landonf


Posts: 86
Registered: 7/23/12
Re: Serial Disk Driver
Posted: Oct 19, 2012 6:28 PM   in response to: techknight in response to: techknight
Click to report abuse...   Click to reply to this thread Reply
techknight wrote:
Man. your moving alot faster than I. hehe.

Once i get this research and development project done for work (which has been months now). I will have free time again to get coding on the hardware.

Did you have any thoughts on the hardware? With SPI for an SD Card, I think we could achieve 57.6kbps line speed with an 8-bit AVR?
bbraun


Posts: 493
Registered: 7/25/12
Re: Serial Disk Driver
Posted: Oct 19, 2012 6:42 PM   in response to: landonf in response to: landonf
Click to report abuse...   Click to reply to this thread Reply
Related to hardware specs, 115kbps and 230kbps may be reasonable from AV macs on. Or if someone really wants, polling in the whole thing with interrupts disabled, and manually configuring the SCC clock (what localtalk does) might work, although that'd be a lot more work and treading on thin ice IMO. But it could achieve higher speeds even on older hardware.

Anyway, just some thoughts when considering hardware design.
landonf


Posts: 86
Registered: 7/23/12
Re: Serial Disk Driver
Posted: Oct 19, 2012 7:09 PM   in response to: bbraun in response to: bbraun
Click to report abuse...   Click to reply to this thread Reply
bbraun wrote:
Related to hardware specs, 115kbps and 230kbps may be reasonable from AV macs on. Or if someone really wants, polling in the whole thing with interrupts disabled, and manually configuring the SCC clock (what localtalk does) might work, although that'd be a lot more work and treading on thin ice IMO. But it could achieve higher speeds even on older hardware.

Anyway, just some thoughts when considering hardware design.

Clocked at 20 MHz, 230kbps should give us about 35ms, or 695 clock cycles per byte, in which we'd need to perform both recv/send via SPI and USART. (Please feel free to double check my math).

It appears that an SD card write can have significant write latency, so we'd need reasonable write buffering, too. I'm not experienced enough to say if that's doable, but it seems so.

Of course, that leaves very little power left-over for doing anything smart; DMA w/ pre-fetching, large write buffering, etc. Maybe it makes sense to go a bit larger than atmega.
bbraun


Posts: 493
Registered: 7/25/12
Re: Serial Disk Driver
Posted: Oct 19, 2012 7:18 PM   in response to: landonf in response to: landonf
Click to report abuse...   Click to reply to this thread Reply
That's another interesting tradeoff. Right now, with the current driver, it really doesn't matter when the response is received as long as it's within about 5 seconds. If it requests 100KB, and you only have an 8KB buffer, making the mac wait while you refill the buffer is totally fine right now. Or even operating without a buffer and writing to serial as they are retrieved would be fine. As long as the protocol is followed, it really doesn't matter if the controller is slow returning the bits. Obviously the faster the bits can be returned, the faster the mac side experience would be, but it's not like we'd be missing interrupts or negatively impacting mouse movement, floppy usage, or use of the other serial port.

But, if taking the localtalk approach and polling in writes with interrupts disabled, making the mac wait would be bad.
landonf


Posts: 86
Registered: 7/23/12
Re: Serial Disk Driver
Posted: Oct 20, 2012 1:25 PM   in response to: bbraun in response to: bbraun
Click to report abuse...   Click to reply to this thread Reply
It seems like ARM µCs are well suited to the task, but I don't know much about them. Most of the development boards I've come across have SPI->MicroSD and UART->RS232 on-board, which is obviously just about all we need for this.

Other than the basic peripheral support we need, picking something that's widely supported (and cheap) seems wise.
dougg3

Posts: 190
Registered: 8/13/12
Re: Serial Disk Driver
Posted: Oct 20, 2012 2:56 PM   in response to: landonf in response to: landonf
Click to report abuse...   Click to reply to this thread Reply
There are ARM MCUs out there with a built-in SD/MMC controller. These should give you faster data transfers than SPI because they use the full four-bit transfer mode. Probably doesn't solve the write speed problem, but it might help with read latency.

One example processor is the NXP LPC178x series. Looks like some of the ST STM32F chips have it too. I'm not sure if there are any cheap dev boards using these processors that already have an SD socket hooked up to the SD controller, but it might be worth some research.
bbraun


Posts: 493
Registered: 7/25/12
Re: Serial Disk Driver
Posted: Oct 21, 2012 6:40 PM   in response to: dougg3 in response to: dougg3
Click to report abuse...   Click to reply to this thread Reply
serialdisk0.7.sit.hqx

The big change here is the control panel works on System 6. Tested on 6.0.7 on a Classic. The driver worked, but the control panel would crash due to differences in System 6 vs. System 7 control panels.
This also handles launching the control panel when the driver hasn't been loaded.
tt


Posts: 144
Registered: 8/25/12
Re: Serial Disk Driver
Posted: Oct 21, 2012 7:22 PM   in response to: bbraun in response to: bbraun
Click to report abuse...   Click to reply to this thread Reply
bbraun, just to add some feedback...I tried the Serial Disk CP, but ran into a couple issues:

  • I couldn't unpack v06 & v0.7 for some reason =/
  • I tried v0.5, but got bombs at startup mentioning "unimplemented trap and restart with extensions disabled" - setup is Basilisk II as built according to landonf a few posts back, except under Snow Leopard (with Mac OS 7.5.5 & 7.1). Does not bomb at start-up in mini vmac.
bbraun


Posts: 493
Registered: 7/25/12
Re: Serial Disk Driver
Posted: Oct 21, 2012 7:55 PM   in response to: tt in response to: tt
Click to report abuse...   Click to reply to this thread Reply
Thanks for giving it a try. I was using Stuffit, and stuffit's compatibility seems... lacking.
I've gone back to using Compact Pro, since that seems to actually work.

serialdisk0.7.cpt.hqx

I'm not sure about the 0.5 thing. I've not been able to get BasiliskII to actually launch and boot anything, so I can't test with it.
bbraun


Posts: 493
Registered: 7/25/12
Re: Serial Disk Driver
Posted: Oct 21, 2012 8:44 PM   in response to: bbraun in response to: bbraun
Click to report abuse...   Click to reply to this thread Reply
It also looks like I broke the stdio option by putting a debugging printf in the Get Size command (the first command issued).

serialserver0.6.tar.gz

This removes that printf and the --stdio option needed for B2 should work again. Sorry about that.
landonf


Posts: 86
Registered: 7/23/12
Re: Serial Disk Driver
Posted: Oct 21, 2012 10:00 PM   in response to: bbraun in response to: bbraun
Click to report abuse...   Click to reply to this thread Reply
bbraun wrote:
It also looks like I broke the stdio option by putting a debugging printf in the Get Size command (the first command issued).

serialserver0.6.tar.gz

This removes that printf and the --stdio option needed for B2 should work again. Sorry about that.

That seems to do the trick. Basilisk II will still occasionally lock up during startup due to a bug in how it handles opening/closing the serial port (to fix, just kill the child process -- the main process is stuck on waitpid()). I'll try to see about actually fixing that bug next.

However, I'm getting I/O errors on System 7.6. The disk mounts, and I can see the files on it, but actually attempted to read or write a file fails (ex: "SimpleText cannot open this document. It may be in use by someone else"). If I check the serialserver log, I don't see anything reported there.
bbraun


Posts: 493
Registered: 7/25/12
Re: Serial Disk Driver
Posted: Oct 21, 2012 11:49 PM   in response to: landonf in response to: landonf
Click to report abuse...   Click to reply to this thread Reply
When you get failing io operations, can you open the control panel and check the connection status?
Back to the whole problem with Mac OS not having a good way to say "disk is gone and not coming back". If for whatever reason the serial connection gets out of sync, the control panel will say "Disconnected".

Some of the things I'm worried about are:
The timeout mechanism. The way the serial drivers work, in this case with reads, is you say "read X bytes", called asynchronously, and the serial driver won't call your completion routine until X bytes have been read. The serial driver has no timeout mechanism internally, the caller implements their own however they want, and you issue a PBKillIO call to terminate any outstanding reads. My timeout is through a VBL task, when an IO routine is called it sets a counter to 1, the VBL task runs about once a second, and increments the counter each time through. When the counter hits a fixed value (7 right now), it terminates the outstanding read and marks connection as bad, and immediately fails all future IO operations. However, there are times when a read (or write) could legitimately take longer than that. If an app issues a large read (I just had SimpleText issue a 250KB read), the read operation will take a long time. Best case at 57.6kbps, the timeout will be hit every time. One solution is to have a dynamic timeout based on link speed and size of the request. Another solution is to break it up into smaller requests (512bytes?) so the timeout gets called. This has both higher system load due to more completion routines, and lower throughput due to higher overhead of the serial protocol and latency in request processing. But it would detect failures soonest.
The serialserver's log should provide some insight here on the size of requests. I should probably also add some timestamps to the serialserver's output...

The other thing I'm worried about is B2's serial driver and requests. B2 replaces the serial driver. This means with the Serial Disk driver, the B2 driver is being called from within another driver, and depends heavily upon the original driver's semantics.
bbraun


Posts: 493
Registered: 7/25/12
Re: Serial Disk Driver
Posted: Oct 22, 2012 12:53 AM   in response to: bbraun in response to: bbraun
Click to report abuse...   Click to reply to this thread Reply
serialdisk0.8.cpt.hqx
serialserver0.7.tar.gz

This adds an adaptive timeout to the serial disk driver, based on the selected baud rate and amount of data in the request, and adds a healthy fudge factor. A timeout should be no shorter than ~5 seconds though.
This also removes the log message of the incomplete writes each time through the write loop, and adds timestamps to the log messages. This is helpful for debugging the timeout issues.
tt


Posts: 144
Registered: 8/25/12
Re: Serial Disk Driver
Posted: Oct 22, 2012 2:14 AM   in response to: bbraun in response to: bbraun
Click to report abuse...   Click to reply to this thread Reply
Unfortunately I was unable to boot with Serial Disk v0.7, the system bombed again while loading the CP at startup. Does Serial Server influence booting?

command: ./serialserver --stdio --disk serialfile --baud 57600
techknight

Posts: 110
Registered: 10/13/12
Re: Serial Disk Driver
Posted: Oct 22, 2012 8:44 AM   in response to: tt in response to: tt
Click to report abuse...   Click to reply to this thread Reply
If you want to go bigger than a Mega, then your taking the project on yourself. I would be out of it.

AVR is the only MCU series that I use, and familiar with. anything else, game over.

Also about timeouts, thats why I mentioned at the very beginning of the entire thought process is using sector-by-sector transfers. 512bytes from the serial. Yes, higher load, but at that point your timeouts would be reasonable, and you could do byte-wise timeouts. if a particular byte isnt received in a particular period, kick it out.

Message was edited by: techknight

Point your RSS reader here for a feed of the latest messages in all forums