c0ntr0l replacement - expressions of interest required

[Luap]

This sounds really good so far j00lz! Especially if & when you get the Mac versions off the ground. Thats gonna make a lot of people very happy

[j00lz]

Yes you are correct, that's the message I meant. It's implementation is buggy (it has no bounds check), and I'm exploiting this "feature" to read past the contents of the pattern data into the upper 2048 bytes of the EEPROM (currently the track data). Simple hacking. Without boring everybody with implementation details, Safebox issues a series of requests for patterns at locations 1 to 8 in banks 17 to 32 (!) and the x0xb0x happily complies and returns whatever is in there, in chunks of 16 bytes.

But yes, ideally, it would be better to have a dedicated command that returns any number of bytes from any position in the EEPROM.

Does n0nx0x modify any of the constants defined in pattern.h? Specifically, does it alter the PATT_SIZE constant? If so, what happens to the track data?

EDIT: If various firmware flavours are going to alter such basic things as the pattern or step structure, then it would be a good idea to implement a command in the communication protocol that allows firmware version identification. That way I can support all firmware versions. I would be happy to define the new command and implement it in the official firmware, if Lady Ada accepts my request to contribute to the sourceforge repository.

in the so called "SafeBox" there _should_ be a way to extract the whole eeprom memory

the "READ_PATTERN" message - if you're talking about the RD_PATT_MSG thing in compcontrol.c - this is not really copying the "whole" eeprom at all
the original x0xb0x firmware stores 128 patterns of 16bytes each, that's 2048 bytes, while the eeprom is 4KB afaik
n0nx0x on the other hand stores again 128 patterns of 21bytes, but will use even more of the eeprom memory for other things
right now, my RD_PATT_MSG will indeed return the 21 bytes of the Nth pattern like it should, so in theory it should be possible to extract a n0nx0x memory file

another problem
c0ntr0l's "backup EEPROM" algo probably "scans" 16 banks by 8 pattern slots because this is how it looks like in compcontrol.c
this is very bad then

a possible solution:
we should implement a sepparate message in compcontrol.c for extracting the EEPROM memory, not patterns!
the new "c0ntr0l" app, whatever it's called should have a sepparate feature for this too

[antto]

yes, in n0nx0x PATT_SIZE is 21
thus when c0ntr0l asks for a pattern - n0nx0x returns 21 bytes of data, logicaly

there is no track data yet, there is no track mode neither
for now there are only 128 patterns in memory
but once i have the new CPU mod, there will be enough room for implementing the new trackmode and other features into the firmware - thus, probably all of the 4KB will be used
also, there is a possibility that i might change the pattern structure again, but i'm not sure yet, we'll see

if your SafeBox app asks for patterns in banks 17 to 32 to get the "unused" portion of the EEPROM - then using this with n0nx0x will probably do something very BAD
it'll try to call spieeprom_read() with invalid addresses, i'm not sure what'll happen but if it was M$ winbooz - it would definately crash or throw a page fault or access violation

i'm afraid not to corrupt the bootloader because i can't fix it (i don't have the tools)

that's why i said there should be a sepparate message for extracting the memory not the patterns

[j00lz]

I'm guessing that the official c0ntr0l app doesn't work with n0nx0x either?

As for trying to read or write past the EEPROM's physical size, yes it could have some interesting effects. You might hear some novel sounds. You might even see some small fireworks and accompanying smoke if you're lucky

For now I will modify Safeb0x so that if it receives more than 16 bytes in response to a RD_PATT_MSG it will stop. So, no fireworks, sorry.

Is the n0nx0x step/pattern structure documented anywhere (source code or otherwise)? I would be happy to support it in safeb0x if I can.

yes, in n0nx0x PATT_SIZE is 21
thus when c0ntr0l asks for a pattern - n0nx0x returns 21 bytes of data, logicaly

there is no track data yet, there is no track mode neither
for now there are only 128 patterns in memory
but once i have the new CPU mod, there will be enough room for implementing the new trackmode and other features into the firmware - thus, probably all of the 4KB will be used
also, there is a possibility that i might change the pattern structure again, but i'm not sure yet, we'll see

if your SafeBox app asks for patterns in banks 17 to 32 to get the "unused" portion of the EEPROM - then using this with n0nx0x will probably do something very BAD
it'll try to call spieeprom_read() with invalid addresses, i'm not sure what'll happen but if it was M$ winbooz - it would definately crash or throw a page fault or access violation

i'm afraid not to corrupt the bootloader because i can't fix it (i don't have the tools)

that's why i said there should be a sepparate message for extracting the memory not the patterns

[antto]

yes, both the official and the "modified" c0ntr0l apps work with n0nx0x right now, since there are only 128 patterns in memory for now
thus 128 * 21 = 2688 bytes
erm, they "work" - i mean they don't try to ask for data beyond 4KB
but in practice - none of them can really extract the memory for an unknown reason

even with the stock 1.05 firmware, and SokkOS - it didn't work:
"Backup EEPROM" outputs just a portion of the memory - it varies every time but it's around 1.85KB (it should be 2048 bytes as i already said)
"Import EEPROM" simply never worked
"Clear EEPROM" same as above

with n0nx0x - "Backup EEPROM" outputs again a ~1.85KB file (while it should be 2688 bytes right now)

the n0nx0x pattern structure - n0nx0x is still in an early stage of developement, and the structure might change
i can post it anyway if you insist

all i really wanted was an app that safely and reliably can import and export the whole EEPROM memory from any OS, even if that means that all popular OS'es need to be modified a little

another possible solution would be to simply provide the user of an editbox where he can type the amount of memory there is on his x0xb0x (since in the future, there might be a memory MOD for more than 4KB)
and simply put a RD_CHUNK_MSG into the firmware, which will return a variable sized portion of memory to the SafeBox app
this way it'll be possible to extract the whole memory even if it's modded, and the responsibility is left to the user, he has to be careful ;]

[j00lz]

What your are asking for is kind of self-contradicting, because it is not "simple" to modify all flavours of the firmware. Sokkos probably does't have enough room left to include the new function. The source code for n0nx0x is not available. Also, this would require all users to upgrade their firmware, again not a "simple" task for some.

What I have implemented may not be the best solution but it works straight away with all firmware variants except n0nx0x. That is the best I could do with what I had, and it will cover most of the user base, since from what I understand n0nx0x is still at the experimental stage and not in widespread use.

People who want to try out n0nx0x can still use Safeb0x before they upgrade to backup their work, and restore it when they "downgrade" to a less experimental and more stable firmware version. This has got to be better than the current state of play, which is "install n0nx0x and loose all your patterns". If anything, it should increase the number of guinea pigs willing to try out n0nx0x'

yes, both the official and the "modified" c0ntr0l apps work with n0nx0x right now, since there are only 128 patterns in memory for now
all i really wanted was an app that safely and reliably can import and export the whole EEPROM memory from any OS, even if that means that all popular OS'es need to be modified a little

another possible solution would be to simply provide the user of an editbox where he can type the amount of memory there is on his x0xb0x (since in the future, there might be a memory MOD for more than 4KB)
and simply put a RD_CHUNK_MSG into the firmware, which will return a variable sized portion of memory to the SafeBox app
this way it'll be possible to extract the whole memory even if it's modded, and the responsibility is left to the user, he has to be careful ;]

[antto]

1) this new "function" is not a big piece of code, it'll probably be a few lines just like the RD_PATT_MSG
2) if an existing firmware has no more room to fit the function - then fine
3) if the user has difficulties flashing new firmware - fine

remember, this is an "additional" function
the SafeBox app will try to tell the x0xb0x to "RD_CHUNK_MSG"
if the firmware does not have that routine - it won't respond and nothing goes wrong

i mean, this is an "addition" which will be available if the users want it

right now it seems it's only me who's asking for it, i'm not trying spread n0nx0x or anything
but in case you didn't know, the new CPU mod is about to be released very soon, and i'm pretty sure there will be even more brand new OS'es born once it gets into more x0xb0xes
and these new firmwares might be even more extreme than n0nx0x, let's try to think for the future, no matter what, there should be at least 1 function to import and export, reliably the whole memory

don't worry about the OS'es and who's gonna modify them to put this new function there - this will come too

and btw, i don't care if people will lose their patterns when switching between n0nx0x or anything else
if it ain't broken - don't fix it

n0nx0x is not "stable" ? how exactly?

[j00lz]

By "not stable" I meant that it is changing... Not that it is buggy (I wouldn't know since I haven't tried it).

also, there is a possibility that i might change the pattern structure again, but i'm not sure yet, we'll see

I am aware of the existence of the new CPU project. I will be getting my grubby hands on one as soon as they get out

Your suggestions are all great. How about we add not one but 2 new commands to the firmware(s) comp. control interface:
- GET_X0X_INFO_MSG: Asks for the b0x to report its vital statistics, such as firmware kind and version number, as well as its CPU kind, and the size of its EEPROM. While at it, you could even add the user's name and some contact details in there, in case the box gets lost or stolen (I'm serious).
- RD_CHUNK_MSG: as you describe. Should allow specification of the chunk size. Should also have bounds checks.

and btw, i don't care if people will lose their patterns when switching between n0nx0x or anything else
if it ain't broken - don't fix it

Shouldn't that be "If it aint broken, don't break it"?

[antto]

yes, it's a good idea
an info message request, the firmware returns it's name, version, capabilities and so on
the chunk message should be in both RD and WR forms, range checks on the firmware side

pattern managing (converting memory files between different OS formats or exporting/importing to software clones) could be even put into a web-based app (java?) or something, that you only need to have a browser and there is no difference whether Mac/Win/Lin or whatever you're running it on
am i right?

i could write win console apps for converting between formats, or the pseudo code for the convertion, but this doesn't really help since there are already a lot of non-win users out there

the x0xb0x could be used in quite different ways too, the bigger CPU opens up more room for experimentation
one could write a sophisticated arpegiator OS, or drum machine
or a midi sequencer / generator
i guess there's quite a lot you can do with it ;]
that's just blurry thoughts but.. you get my point..

[darffader]

pattern managing (converting memory files between different OS formats or exporting/importing to software clones) could be even put into a web-based app (java?) or something, that you only need to have a browser and there is no difference whether Mac/Win/Lin or whatever you're running it on

unless someone is quicker than me, that functionality is part of something i'm working on (php/javascript, no java). It will take a few more months though due to other stuff

[Paradigm X]

- K0ntr0l: As above, but with many many more features. Import of patterns from ABL (.pat) and Rebirth (.rbs) is supported. I will add support for other file formats in the future, depending on user feedback. Export is currently only to ABL and MIDI. Again, users will decide on what formats get implemented next. Multiple patterns can be imported and assigned to consecutive banks/positions with just a few mouse clicks. Patterns that exceed the 16 steps limit are automatically split and assigned to consecutive positions. Patterns can be named. Sets of patterns can be defined and recalled with ease as well. It is basically a pattern librarian and editor, with full x0xb0x connectivity. It is possible to push modified (or altogether different) patterns to the x0xb0x while the box is playing, I have been having a lot of fun with this feature . Multiple pattern editors will be provided (text, graphical...etc). This application will be released in the coming weeks, as shareware. It will be very reasonably priced. Again, support for Mac OS X is pending resolution of the issue mentioned above. I'm pretty sure it will come eventually but I can't say when...

Reading this again, this is going to be awesome. Im sure a lot of rebirth/ABL users might be interested too.

The only other format which is likely to be of use is the d16 PHOSCYON, but im not a user so unsure how the patterns are stored. Still a fairly basic text based scheme AFAIK.

Ill def post about it on the ABL forums once its done.

Just one question; not really an issue either way' is K0ntr0l going to also be able to send firmware as per the original c0ntr0l.

Excellent stuff.

[Paradigm X]

http://www.d16.pl/forum/viewtopic.php?f=9&t=8&start=15

Info on d16 pattern format. FYI.

[j00lz]

http://www.d16.pl/forum/viewtopic.php?f=9&t=8&start=15

Info on d16 pattern format. FYI.

Funny you should mention this I downloaded the Phoscyon demo this morning, and spent the afternoon implementing and testing Phoscyon pattern/bank file reading and writing. It was a piece of cake as it was all XML and almost exactly matched the internal Kontrol pattern/step representation. So Kontrol will support that format as well. Also, using Kontrol it will be possible to convert between ABL and Phoscyon pattern formats.

Re: firmware update, I'll add that to the list of TODOs. It's not at the top, though, as I am focusing more on pattern management and interoperability.

[Paradigm X]

Cool.

As i said i dont actually use it myself just thought it would maximise your potential customers - this thing really does sound very helpful for me so hope you get something back for it.

What did you think of phoscyon out if interest, i dled the demos of ABL and this ages ago and went with ABL in the end. Heard the latest versions very good indeed. Although with a x0x and ABL i really dont need another clone (just a real one).



Firmware, no biggy, want to try n0nx0x but happy with s0kk0s, happy to use control for now, although it would be a nice addition its by no means critical as im sure people dont change firmware very often. the pattern management etc is whats most useful. I assume kontrol will have all the features of safeb0x as well.

Nice one, look forward to the main release.

[j00lz]

Things are coming along nicely...

All the underlying framework implementation (x0xb0x comms, ABL/Phoscyon/Rebirth file support) is done and pretty much fully tested. I am currently implementing a piano-roll control for pattern display and editing. Also working on the pattern library features, but I think I might do a release before this is finished, because K0ntr0l is already quite usable as it stands.

I'm attaching a screenshot of the main screen as it currently stands. It is very likely that the layout will change significantly before release, but it gives an idea of the features so far.

K0ntr0l screenshot

PrototypeMainScreen.png (57.92 KiB) Viewed 2995 times