New Firmware project

Discuss mods, hacks, tweaks, etc.

Moderators: altitude, adafruit_support_bill, adafruit, phono, hamburgers

Please be positive and constructive with your questions and comments.
User avatar
helyx525
 
Posts: 61
Joined: Fri Apr 23, 2010 11:58 am

Re: New Firmware project

Post by helyx525 »

3phase wrote:is it possible to just replace the cpu chip with another one? is ther a type that is pin compatible and just has more memory?
unfortunately, no. There is no larger AVR cpu with the same pinout. Short of re-designing the mainboard, the simplest approach is to use daughterboard and a ribbon cable because there isnt enough room to socket a daughterboard in place of the old CPU.
editing while runing is not possible with the real 303.. but that is something that shouldnt be portet to the xox... it should be rather optimize to go easyly into edit/write mode without loosing external sync..
Agreed.
tab input is vitaly important IMO to generate lines that are original and express your musical will...
especially on chained multibar patters tab in safes a lot of time..
if its absolutly impossible to get all this functionality within 16k i defenetly would opt for a hardware mod with better cpu..
As antto also stated, there just isnt enough space in 16k to get a good sequencer written.
i suggest that booth roads are developed.. a as compact as possible 16 k version that abandons some xox features.. but surly not midi... and a hardware mod that allows a real os upgrade that actualy might give the xox a superior state over the real 303...
I really think the upgraded cpu is the only way. If people want to to upgrade their sequencer, they need to get the CPU mod. I want it to be a simple $30-$40 part that you just pop the old CPU and plug in the new. Then they can flash the new firmware if it didnt already come loaded when they bought the mod.

Thats why its crucial that we get the firmware hackers together in my "new cpu project" thread. We need everyone to get their requirements out there and to agree on the CPU so we dont ever have to do this again. I dont want there to be 7 different cpus and firmwares out there. Old vs new CPU will be painful enough.

User avatar
helyx525
 
Posts: 61
Joined: Fri Apr 23, 2010 11:58 am

Re: New Firmware project

Post by helyx525 »

A lot of non-technical users are complaining about how tedious firmware updates are.
The process needs to be simpler.

Feature request:
On the new bigger CPU, we should consider supporting firmware updates via midi sysex.

That means firmware developers would need a utility to convert their .hex file into a sysex .mid file with appropriate pauses for block flashing. I can bang that out in c++ in about a day. It also means we will need to add MIDI I/O to the bootloader and define a sysex format for sending the firmware in blocks.

User avatar
helyx525
 
Posts: 61
Joined: Fri Apr 23, 2010 11:58 am

Re: New Firmware project

Post by helyx525 »

double post.. oops
Last edited by helyx525 on Tue May 18, 2010 8:53 am, edited 1 time in total.

User avatar
antto
 
Posts: 1636
Joined: Thu Apr 15, 2010 3:21 pm

Re: New Firmware project

Post by antto »

i understand that both the bootloader and the firmware are on the same EEPROM memory on the CPU, just that the bootloader provides a way to rewrite the firmware part of memory, without overwriting the bootloader itself (this would be dangerous if it goes wrong)
but how do you overwrite the bootloader then? in case you want to put a different one?
afaik (sorry for being so unfamiliar) the EEPROM is programmed with some "device" which (hehe) i don't think i have...
is it possible for mortals like me to switch from the old bootloader to the new one?

User avatar
helyx525
 
Posts: 61
Joined: Fri Apr 23, 2010 11:58 am

Re: New Firmware project

Post by helyx525 »

antto wrote:i understand that both the bootloader and the firmware are on the same EEPROM memory on the CPU, just that the bootloader provides a way to rewrite the firmware part of memory, without overwriting the bootloader itself (this would be dangerous if it goes wrong)
but how do you overwrite the bootloader then? in case you want to put a different one?
afaik (sorry for being so unfamiliar) the EEPROM is programmed with some "device" which (hehe) i don't think i have...
is it possible for mortals like me to switch from the old bootloader to the new one?
Its not hard for a programer to do in 2 stages.
Stage 1, use the bootloader to put in a firmware that acts as a secondary bootloader.
Stage 2, use the secondary bootloader code in the low flash to update the real bootloader in the high flash.

Then you re-flash the main firmware to get rid of the secondary bootloader.

User avatar
rv0
 
Posts: 395
Joined: Tue Jul 14, 2009 4:50 pm

Re: New Firmware project

Post by rv0 »

Helyx525 wrote: Stage 1, use the bootloader to put in a firmware that acts as a secondary bootloader.
Stage 2, use the secondary bootloader code in the low flash to update the real bootloader in the high flash.
Stage 3: realize you've made a mistake somewhere and get stuck in the chicken/egg problem :wink:

User avatar
helyx525
 
Posts: 61
Joined: Fri Apr 23, 2010 11:58 am

Re: New Firmware project

Post by helyx525 »

darffader wrote:Stage 3: realize you've made a mistake somewhere and get stuck in the chicken/egg problem :wink:
:lol: Yeah thats when you start looking for a JTAG cable.

User avatar
rv0
 
Posts: 395
Joined: Tue Jul 14, 2009 4:50 pm

Re: New Firmware project

Post by rv0 »

feature and control wise, this is a quick rendering of the control remapping i did last week.
Image

The design is by no means definite, i just did it to get a basic idea, also the fonts haven't even been aligned well, my CAD software renders it all choppy.
I was planning on printing that design on adhesive silver vinyl and laminate it (still awaiting pricequote for that), but for now it's just a paper overlay on my x0x :lol:

ps: the n0nx0x logo is all about antto's firmware, which he called n0nx0x, because it's a version of his n0nseq :) I just placed it on there because I wanted to see how that 303 type font looks like :)

The other parts of the front panel are left unchanged so far, mainly because there are different kinds of front panels with extra buttons.
Maybe a second adhesive can be made for around the 16 position rotary switch

Meanwhile.. I'm quite happy with the mapping of the buttons like that.. It works really natural and intuitive

3phase
 
Posts: 203
Joined: Wed Apr 22, 2009 2:06 pm

Re: New Firmware project

Post by 3phase »

So we collect ideas for the new firmware here? i think it has allready started so lets keep it here?

I bad guy messed up the cpu thread with some demands regarding clocking...

while it seems to be agreed on that a new 256k flash cpu is the way to go for the future i dont think it hurts to theorize about an ideal OS for the xox..

In a later step this should be streamlined and evaluted for cost/effort ratio..

However...

Ideal clocking in 21th century hardware sequencers

part 1... brainstroming


features one like to see that is working in allways changing hard and software setups in studios and on stages...

1) the clocking and its handling should be apllied on rootlevel of the OS.. all other functions second to that and not interfearing with it.. clock priority..

the elektron machine drum is an example of a sequencer that handles the clock with priority tight, but has "problems" with the execution of its own sequencer.. in reality this results in a rather groovy sounding tool one can rely on as clock master or slave.. a good example that glitches in the sound creation actually can be beneficial while errors in the internal process timing are usually distracting..see doepfer MAQ sequencer as negativ example here...

2) low latency operation.. regardless of what fancy features are applied, a modern machine should be able to be fast on external clock..
And it shouldnt loose a single clock tick...
There is an absolute amount of clockticks in a given time.. If internal processes need time, the sequencer should be able to lock back to its absolute position along the timeline afterwards..

3) the plain clocking by the midi ticks should have priority and the handling of realtime midi commands should interupt all other incoming midi streams ..or handled in seperate buffers wher the realtime clock buffer is allways executed first...

4) it should be possible to have clock/midi thru on root level.. with or without merge of the generated midi notes? i know this is a questionable feature because it can eat to much processing power and therfore interfears with the other demands ... however.. would be a handy tool in certain stage situations.. if the clock can be only applied in such a scenario with an offset delay it still might be usefull...

there should be midi out On/Off options for clock, notes and controlers regardless in what sync/master mode the maschine is running..


5) offset delays... an important aspect..overseen by many but.. the truth is..any machine on the market starts at a different time regarding to the incoming midi clock/start command..

You cant run a korg electribe es together with an mfb 522.. they are such a big amount off..the mfb is so early..the electribe so late...
That is not a new thing..even under din sync we know that problem.. a 909 and a 808 and a 303 dont go togehter too well.. you need the 909 as midi clock master and a half tick delayed korg kms 30 to sinc the 808 while the 303 is synced directly by the 909..than its fine...

some people might have another opinion here.. but in reality more and more producers buy clock shifters to get their machines in line..

with the xox we have a rather good compromise.. its faster than the original 303.. but not too earlie..still a bit late...but as slave it sounds totaly different as master... as master its earlier..and thats a nice feel too.. i like to have booth options without changing my sync setup

I wish for all modern sequencers the ability to shift the event timing..
because shifting earlier is difficould.. or??

the internal event timing should be very fast..much less than 1ms... but

WITH an OFFSET delay..fine adjustable in 0,1 ms steps at least 5 ms later.. maybe 64 steps later...and 64 steps earlier? than the whole machine starts one bar later?

a point to discuss.. but i vote for an offset parameter to allow a desired feel.. as a system setting

user interface idea:
a midi setup page where one can dial the offset with graphical indication in the 16 leds..while the key leds show which offset range is actually dialed.. so maybe e key highlighted no step led is zero offset,,,while f key and all 16 steps lid is = 32 steps = 3,2 ms late.. ...
other keys like the transpose switches switch midi out of notes and clock... something like that..or better
...


6) shuffle.. while some think that shuffle can be applied by shifting the ppq clock i strictly opt to implement shuffle in the way its done in the other sequencer designs on the market..as a delay setting on actual sequencer steps..after the global offset delay.. just applied to certain steps..

Look at the elektron machine drum..that is a pretty good example how shuffle is implemented. as a sequencer that controls an event delay

There the shuffle has an own sequencer track..where each step set delay on/off for that step.
So when you set all steps you move the machinedrums drum sequencer later.. when you only set the odd steps we have normal shuffle and when you set all even steps you have negative shuffle..

this just as an example how the mechanics work there.. dont need such a feature,,the xox is no drummachine... but.. shuffle is an interpretation that is happening on top of a steady clock and not as the result of a modulation of that clock..

however.. a natural shuffle that is based on a real sine lfo modulation ca be interesting aswell..something that is not happening between 16th notes.. on a longer scale..over 4 bars or so..

anyway.. the unit should be able to perform shuffled grooves.. in ideal with an remotable shuffle amount via midi controler... so multiple units can be shuffled at once..

shuffle settings should be cpmpatible with the standards.. so a 63% shuffle is the same as on other machines..

user interface idea:
shuffle needs to be adjustable while running... i wouldnt say thats necessary to be stored with the pattern.. might be nice if.. but its rather something that is a performance parameter...
so it shoul be accesible while running..maybe with a double keystroke the keyboard key allow the input of 12 defined shuffles? maybe in conjunktion with the tempo dial for fine adjustments?
so actually other effects might be selected ther as well..with fine adjust actually 3 or 5 shuffel setings would be enough to cover the whole range with little moves... so 5 coarse steps with fine adjustment..

The shuffle value should be kept on pattern changes... Or? maybe in one of the user pages one can select the behavior how the xox handles shuffle and transpose on pattern changes? or the possebility to decide in edit mode how the pattern behaves.. either on select it uses the pattern settings for shuffle and rests transpose
or it takes the values of transpose and shuffle from the previous settings?

would be maybe good to use the exsisting user pages for such setup things... maybe even while running...

maybe it would be good to have a key that while pressed allows to dial the mode select button and its only read out when you release that function key?
7) start performance.. refference akai mpc 3000.. this machine starts imeadeatly.. as master..and in external sync...

the trick here is that the machine dont waits for clock pulses or tempo information..it just starts imeadeatly on the start BANNED recieved..and than starts the calculation.. within the first quarter note its locked..

result... you can play the mpc 3000 as a drumer with a trigger pad.. and it just starts exactly when you hit the pad and not some little time later.. that is big fun.. and actually one thing that should be implemneted in any modern sequencer..especially when the xox acts as master that is beneficial...
But also when the mpc 3000 is slave you can stop it at any time and start it by hand again..it will start imideatly without waiting for a clockpulse.. what actually results in a situation that with an mpc 3000 that actually works, and you dont need 3 trys or have to restart the master clock to get it tight.
Most machines on the market are pretty unprdictable when restarting them again while being clock synced because they wait for the next tick to start.. or even worse..wait for theire own internal get ready routines and than wait for the next incoming clock...

the mpc 3000 uses the set tempo as start tempo..so when master clock tempo and machine internal tempo are maching you dont even get that the first quarter note is a bit jumpy..

in any way this behavior sound musical.. a drummer that starts playing is also a bit jumpy on the first hit..but usually never too late... The mpc 3000 is really outstanding regarding this..and maybe the mpc 60 aswell..never measured an mpc 60 in such great detail as i did with my mpc 3000..modern mpc´s are not this good anymore regarding that details.. at least the mpc 2000 wasnt... i havent tetsted any later mpc´s


7.2) when the xox is sync slave it would be nice when as in the akai mpc 3000 any start BANNED is performed as described.. but.. an option would be nice..where starts are quantized to the next one..
therfore the modern sequencer should be able to read song position pointer if available..
that comes in handy when the xox acts as din sync interface for macines that need to be stopped to enter the write mode ( 808 for example ) . that should be a global setting...
in any case the xox should be able to act as din sync master while being a mid clock slave...
in ideal with an own offset delay for the din sync output.. that would be perfect to act as 808 sub master... i reality din sync machines are usually earlier than theire modern midi brothers... so if the dinsync out while beeing a slave needs a little process delay it wouldnt hurt anyway...

8 ) dejittering external clocks.. of cause we dont work with jittering clocks..but many do..

again the mpc 3000 as an example..this machine has its legendary name as time keeping tight running machine not at last because it actually is dejittering incoming clocks...

So its actually tighter than the clock it recieves.. simple trick.. as soon this machine has set its tempo after its imeadeate start..after the first quarter note.. it applies an smoothing of maybe 1 bar ( maybe a second?) to the incoming clocks... you can test that by applying tempo changes to the mpc 3000..it reacts late..with a lag.. cant tell exactly how big the lag is.. but its easy to get...this machine is dejittering...no imideate followig on any little tempo fluctuation...not bad sounding ..rather musical..

So it starts imeadeatly.. gets the tempo wright within 1 quarter note.and afterwards is dejiitering incoming clock and reacts musicaly lagged to tempo changes..

applause to Roger Linn.. great design. the ideal behaviour of a slaved sequencer..already concieved in the 80´s..

Its a total musical missconception that a clock slave has to follow the clock like a slave in realtime. thats just a primitiv because simple algorythm to handle incoming clock..actually no algorythm..just 1:1...
And so any usb related jitter is transported to the musical system.. instead just acting musicaly.. no band is performing a tempo change within a 16th note.. a lag on a tempochange is just natural..

the main reason why ableton live is bad as clockslave while native instruments reaktor is perfect..

try tempo changes on reaktor and you will see the lag

************************************

So..that it was for now..some ideas regarding clocking.. this of cause would take quite some coding..
But Roger Linn has seen sense in that and gave his drumcomputers more than just primitv clock following..

Even if that is not the most desired thing or most important enhancement for a new OS..
it wouldnt hurt to have highend clocking algorythms in the xox..
problem..when this is nt planed from the beginning its maybe not possible to implement it later without rewriting everything

So...my main suggestion..

At least implement a clocking system.. on the root of the new os ..that allows later modification and enhancements in the mentioned directions..

the os should have something like modules.. where the first one is clock handling.. in conjunction with porthandling.. and on top of that the sequencer... and booth of theese fuctional blocks should have designed space for later enhancements..

because all sequencers relate on clock handling this should be handled as a seperate machine inside the machine.. and the actual i/o..editing sequencer style on top of that...

so people could have a 303 style sequencer ..but all core xox functionality.. or an original xox sequencer..with all core enhancements of the new os... or even turn the xox into a drumcomputer without rewriting everything...

my 25 cents ;-)
Last edited by 3phase on Thu May 20, 2010 10:56 pm, edited 6 times in total.

User avatar
phono
 
Posts: 1502
Joined: Wed May 02, 2007 4:01 pm

Re: New Firmware project

Post by phono »

3phase wrote:a 909 and a 808 and a 303 dont go togehter too well.. you need the 909 as midi clock master and a half tick delayed korg kms 30 to sinc the 808 while the 303 is synced directly by the 909..than its fine...


i dont have this problem with mine, but i run them all on sync from an msy2 (early revision 909 with midi bug so i dont use midi) also pretty sure the 909 sync is in only not out

3phase
 
Posts: 203
Joined: Wed Apr 22, 2009 2:06 pm

Re: New Firmware project

Post by 3phase »

phono wrote:
3phase wrote:a 909 and a 808 and a 303 dont go togehter too well.. you need the 909 as midi clock master and a half tick delayed korg kms 30 to sinc the 808 while the 303 is synced directly by the 909..than its fine...


i dont have this problem with mine, but i run them all on sync from an msy2 (early revision 909 with midi bug so i dont use midi) also pretty sure the 909 sync is in only not out

you are wright..that was moded on my 909.. i also had booth midi outs with clock out and a tuneable hihat....
it was anyway just an example that the way you connect the machines effect the way they sound together because the soundengine is not allways quick on the clock ticks... the 808 is exeptinally fast for example and not for all styles and tempos its nice when the kick is earlier than the rest of the tracks..
i allways need an extra midi interface to delay my 808 on stage...

different situation with the 303... i often shift 303 recordings earlier in the daw.. xox as master has a much earlier feel than xox as slave.. booth is good for certain styles or actualy effects the style of lines you write..

its a matter of taste.. and therefore a good thing to be adjustable..
A luxury feature.. but one to think off in the structural design..

maybe i am wrong..dont know anything about coding avr´s.. but might be better to think about such structural things at an early stage..

as said earlier..most important for me is better writeability of the sequencer and tab input.. thats the number one wishes i have.. than triplets and chained patterns+editing..

next smart clocking/shuffle options..

and than? next one please ;-)

User avatar
helyx525
 
Posts: 61
Joined: Fri Apr 23, 2010 11:58 am

Re: New Firmware project

Post by helyx525 »

3phase wrote:So...my main suggestion..
At least implement a clocking system.. on the root of the new os ..that allows later modification and enhancements in the mentioned directions..

the os should have something like modules.. where the first one is clock handling.. in conjunction with porthandling.. and on top of that the sequencer... and booth of theese fuctional blocks should have designed space for later enhancements..
I have been thinking about this a lot. I agree that the tempo/transport controls should be a fundamental subsystem within the OS. Things like clock smoothing and shuffle need to be handled consistently, no matter what custom sequencer apps people write. It needs to be done right so nobody else needs to mess with it.

As far as modules go, I have been looking at different RTOS implementations. I like this one because it is small and light weight:
http://www.femtoos.org/
But until I get the hardware to experiment on, I wont decide on something like that.

I have been considering a message based architecture, where there is a central message queue at the core of the system. An application symply loops reading messages from this queue and responding to them. Its similar to the message pump at the center of a windows application. A message could be any external or internal stimulus to the application like a button press or an incoming midi note or a clock tick from the tempo subsystem. There would be ways to restrict what messages are entering the main message queue, such as midi messages on other channels.

As was suggested, tempo and transport events can be given priority and processed first before other messages in the queue.

This would would make writing xoxbox apps much simpler and more modular, allowing us to attach them each to a Mode Switch slot and mix and match the apps we like.

An app would look something like this:

Code: Select all

void my_arpeggiator()
{
   // ask for 1/16th note and start/stop messages
   tempo_subscribe_messages( TMSG_TICK_16TH | TMSG_TRANSPORT );

   // ask for all button presses
   switch_subscribe_messages( SMSG_ALL );

   // ask for midi messages related to the current channel like notes or CCs
   midi_subscribe_messages( MIDIMSG_CUR_CHANNEL );

   X0xMessage* msg;
   while(1) {
      // get the next message, or block until one becomes available
      msg = msgque_get_next_message();
      if (msg->type == MSG_QUIT)
         break;

      // dispatch the message to the appropriate handler
      switch (msg->type) {
      case MSG_BUTTONDOWN:
         // hook front panel button presses
         myapp_on_btndown(msg);
         break;

      case MSG_TEMPO_START:
         // handle arpeggiator start
         myapp_on_start();
         break;

      case MSG_TEMPO_STOP:
         // handle arpeggiator stop
         myapp_on_stop();
         break;

      case MSG_TEMPO_TICK_16TH:
         // handle 16th note step for my arpeggiator
         myapp_on_step();
         break;

      case MSG_MIDI:
         // handle midi note on/off for my arpeggiator
         myapp_handle_midi(msg);
         break;
      }

      // advance queue to the next message
      msgque_skip_message();
   }

   tempo_unsubscribe_messages( TMSG_ALL );
   switch_unsubscribe_messages( SMSG_ALL );
   midi_unsubscribe_messages( MIDIMSG_ALL );
}
Last edited by helyx525 on Thu May 20, 2010 11:31 pm, edited 1 time in total.

3phase
 
Posts: 203
Joined: Wed Apr 22, 2009 2:06 pm

Re: New Firmware project

Post by 3phase »

just one add to the dejittering of clocks..while this is on the midi input beneficial its probably useless on din sync input... assuming that the din sync comes from an old drummachine as master that is so early that no delays are allowed... or an audio clock that is tight allready or might might use desired fancy modulations...

so when on root level we have the midi handling ...and that clock output is applied to the din sync output and the sequencer... din sync input would be something like an insert in between the midi handling block and the actual 303/xox sequencer...


that would have the advantage that din sync operation is direct .. and can act as an alternativ setting.. .. so offset delay and lagged tempo changes via midi..also allowing pc users to have rock solid timing on a jittery midi clock... and just by switching to din sync slave having super fast locktimes and fast tempo changes or audioclocking experiments accesible without changing any settings...

3phase
 
Posts: 203
Joined: Wed Apr 22, 2009 2:06 pm

Re: New Firmware project

Post by 3phase »

sorry to open my mouth again.. just having a sleepless night.. too much energy drinks :-/

however one more idea..usefull or not the coders have to decide,,

about memory format... maybe ..in case there is a bigger memory having two memorys for each pattern...

a main part that stays compatible with actual xoxes.. and a sub memory to store pattern related data of specialized sequencers... like shuffle settings for example..

in conjunktion with this the earlier mentioned idea with maybe having 2 edit pages ..

And just one ectra feature for an expanded sequencer.:

special rnadom step accent and slide events... and octave up down events...

theese special events are conected to a random generator..with an external midi controler like mod wheel or the tempo encoder the probabilty of that special events to happen can be set...


so for exampe if you have the random accent and octve up on a specific note set... mod wheel full up will give you allways accent and octve up on this step.... mod wheel in the middle will give you a 50% probability of that step to be played an octave up and accented... mod wheel down the patern plays normal all the time...

not vey important.. but just got inspired by reading again Helyx idea of adding accents by aftertouch...

such random steps also would allow to alter the sequences without changing the pattern...

anyway.. I try to sleep now and keep more ideas for a later point--first mr Helyx needs his xoxbox..

best regards..

User avatar
helyx525
 
Posts: 61
Joined: Fri Apr 23, 2010 11:58 am

Re: New Firmware project

Post by helyx525 »

3phase wrote: Re: pattern memory
I think it will be less complicated to have each application maintain its own separate patterns and space in the flash memory.
theese special events are conected to a random generator..with an external midi controler like mod wheel or the tempo encoder the probabilty of that special events to happen can be set...

so for exampe if you have the random accent and octve up on a specific note set... mod wheel full up will give you allways accent and octve up on this step.... mod wheel in the middle will give you a 50% probability of that step to beplayed an octave up and accented... mod wheel down the patern plays normal all the time...
This sounds almost similar to another of my back burner ideas where each step of the pattern has a midi velocity value. Currently midi velocity=127 is the only way to trigger an accent. Imagine if the notes in your pattern had all different velocities and they you could control (with a midi CC) the threshold where velocity triggers an accent.
MODWHEEL=0, no notes are accented
MODWHEEL=1, only notes w/velocity = 127 are accented
MODWHEEL=20, notes with velocity > 107 are accented.
MODWHEEL=64, notes with velocity > 64 are accented.
MODWHEEL=127, all notes are accented.

So depending on your arrangement and variation of note velocities in a single pattern, you would have a wide range of expressive accent patterns ranging from no accents to 1-2 accents up to all accents.

Something similar could be fun to experiment with slides and octaves as well.

Locked
Please be positive and constructive with your questions and comments.

Return to “x0xm0dz”