1Midi Kit design 2=============== 3 4The Midi Kit consists of the midi_server and two shared libraries, 5libmidi2.so and libmidi.so. The latter is the "old" pre-R5 Midi Kit and 6has been re-implemented using the facilities from libmidi2, which makes 7it fully compatible with the new kit. This document describes the design 8and implementation of the OpenBeOS midi_server and libmidi2.so. 9 10The midi_server has two jobs: it keeps track of the endpoints that the 11client apps have created, and it publishes endpoints for the devices 12from /dev/midi. (This last task could have been done by any other app, 13but it was just as convenient to make the midi_server do that.) The 14libmidi2.so library also has two jobs: it assists the midi_server with 15the housekeeping stuff, and it allows endpoints to send and receive MIDI 16events. (That's right, the midi_server has nothing to do with the actual 17MIDI data.) 18 19-------------- 20 21Ooh, pictures 22------------- 23 24The following image shows the center of Midi Kit activity, the 25midi_server, and its data structures: 26 27 |image0| 28 29And here is the picture for libmidi2.so: 30 31 |image1| 32 33Note that these diagrams give only a conceptual overview of who is 34responsible for which bits of data. The actual implementation details of 35the kit may differ. 36 37-------------- 38 39Housekeeping 40------------ 41 42- The design for our implementation of the midi2 "housekeeping" 43 protocol roughly follows `what Be did <oldprotocol.html>`__, although 44 there are some differences. In Be's implementation, the BMidiRosters 45 only have BMidiEndpoints for remote endpoints if they are registered. 46 In our implementation, the BMidiRosters have BMidiEndpoint objects 47 for *all* endpoints, including remote endpoints that aren't published 48 at all. If there are many unpublished endpoints in the system, our 49 approach is less optimal. However, it made the implementation of the 50 Midi Kit much easier ;-) 51 52- Be's libmidi2.so exports the symbols "midi_debug_level" and 53 "midi_dispatcher_priority", both int32's. Our libmidi2 does not use 54 either of these. But even though these symbols are not present in the 55 headers, some apps may use them nonetheless. That's why our libmidi2 56 exports those symbols as well. 57 58- The name of the message fields in Be's implementation of the protocol 59 had the "be:" prefix. Our fields have a "midi:" prefix instead. 60 Except for the fields in the B_MIDI_EVENT notification messages, 61 because that would break compatibility with existing apps. 62 63Initialization 64~~~~~~~~~~~~~~ 65 66- The first time an app uses a midi2 class, the 67 BMidiRoster::MidiRoster() method sends an 'Mapp' message to the 68 midi_server, and blocks (on a semaphore). This message includes a 69 messenger to the app's BMidiRosterLooper object. The server adds the 70 app to its list of registered apps. Then the server asynchronously 71 sends back a series of 'mNEW' message notifications for all endpoints 72 on the roster, and 'mCON' messages for all existing connections. The 73 BMidiRosterLooper creates BMidiEndpoint objects for these endpoints 74 and adds them to its local roster; if the app is watching, it also 75 sends out corresponding B_MIDI_EVENT notifications. Finally, the 76 midi_server sends an 'mAPP' message to notify the app that it has 77 been successfully registered. Upon receipt, BMidiRoster::MidiRoster() 78 unblocks and returns control to the client code. This handshake is 79 the only asynchronous message exchange; all the other requests have a 80 synchronous reply. 81 82- If the server detects an error during any of this (incorrect message 83 format, delivery failure, etc.) it simply ignores the request and 84 does not try to send anything back to the client (which is most 85 likely impossible anyway). If the app detects an error (server sends 86 back meaningless info, cannot connect to server), it pretends that 87 everything is hunkey dorey. (The API has no way of letting the client 88 know that the initialization succeeded.) Next time the app tries 89 something, the server either still does not respond, or it ignores 90 the request (because this app isn't properly registered). However, if 91 the app does not receive the 'mAPP' message, it will not unblock, and 92 remains frozen for all eternity. 93 94- BMidiRoster's MidiRoster() method creates the one and only 95 BMidiRoster instance on the heap the first time it is called. This 96 instance is automatically destroyed when the app quits. 97 98Error handling 99~~~~~~~~~~~~~~ 100 101- If some error occurs, then the reply message is only guaranteed to 102 contain the "midi:result" field with some non- zero error code. 103 libmidi2 can only assume that the reply contains other data on 104 success (i.e. when "midi:result" is B_OK). 105 106- The timeout for delivering and responding to a message is about 2 107 seconds. If the client receives no reply within that time, it assumes 108 the request failed. If the server cannot deliver a message within 2 109 seconds, it assumes the client is dead and removes it (and its 110 endpoints) from the roster. Of course, these assumptions may be 111 false. If the client wasn't dead and tries to send another request to 112 the server, then the server will now ignore it, since the client app 113 is no longer registered. 114 115- Because we work with timeouts, we must be careful to avoid 116 misunderstandings between the midi_server and the client app. Both 117 sides must recognize the timeout, so they both can ignore the 118 operation. If, however, the server thinks that everything went okay, 119 but the client flags an error, then the server and the client will 120 have two different ideas of the current state of the roster. Of 121 course, those situations must be avoided. 122 123- Although apps register themselves with the midi_server, there is no 124 corresponding "unregister" message. The only way the server 125 recognizes that an app and its endpoints are no longer available is 126 when it fails to deliver a message to that app. In that case, we 127 remove the app and all its endpoints from the roster. To do this, the 128 server sends "purge endpoint" messages to itself for all of the app's 129 endpoints. This means we don't immediately throw the app away, but we 130 schedule that for some time in the future. That makes the whole event 131 handling mechanism much cleaner. There is no reply to the purge 132 request. (Actually, we *do* immediately throw away the app_t object, 133 since that doesn't really interfere with anything.) (If there are 134 other events pending in the queue which also cause notifications, 135 then the server may send multiple purge messages for the same 136 endpoints. That's no biggie, because a purge message will be ignored 137 if its endpoint no longer exists.) 138 139- As mentioned above, the midi_server ignores messages that do not come 140 from a registered app, although it does send back an error reply. In 141 the case of the "purge endpoint" message, the server makes sure the 142 message was local (i.e. sent by the midi_server itself). 143 144- Note: BMessage's SendReply() apparently succeeds even if you kill the 145 app that the reply is intended for. This is rather strange, and it 146 means that you can't test delivery error handling for replies by 147 killing the app. (You *can* kill the app for testing the error 148 handling on notifications, however.) 149 150Creating and deleting endpoints 151~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 152 153- When client code creates a new BMidiLocalProducer or 154 BMidiLocalConsumer endpoint, we send an 'Mnew' message to the server. 155 Unlike Be's implementation, the "name" field is always present, even 156 if the name is empty. After adding the endpoint to the roster, the 157 server sends 'mNEW' notifications to all other applications. Upon 158 receipt of this notification, the BMidiRosterLoopers of these apps 159 create a new BMidiEndpoint for the endpoint and add it to their 160 internal list of endpoints. The app that made the request receives a 161 reply with a single "midi:result" field. 162 163- When you "new" an endpoint, its refcount is 1, even if the creation 164 failed. (For example, if the midi_server does not run.) When you 165 Acquire(), the refcount is bumped. When you Release(), it is 166 decremented. When refcount drops to 0, the endpoint object "deletes" 167 itself. (So client code should never use an endpoint after having 168 Release()'d it, because the object may have just been killed.) When 169 creation succeeds, IsValid() returns true and ID() returns a valid ID 170 (> 0). Upon failure, IsValid() is false and ID() returns 0. 171 172- After the last Release() of a local endpoint, we send 'Mdel' to let 173 the midi_server know the endpoint is now deleted. We don't expect a 174 reply back. If something goes wrong, the endpoint is deleted 175 regardless. We do not send separate "unregistered" notifications, 176 because deleting an endpoint implies that it is removed from the 177 roster. For the same reason, we also don't send separate 178 "disconnected" notifications. 179 180- The 'mDEL' notification triggers a BMidiRosterLooper to remove the 181 corresponding BMidiEndpoint from its internal list. This object is 182 always a proxy for a remote endpoint. The remote endpoint is gone, 183 but whether we can also delete the proxy depends on its reference 184 count. If no one is still using the object, its refcount is zero, and 185 we can safely delete the object. Otherwise, we must defer destruction 186 until the client Release()'s the object. 187 188- If you "delete" an endpoint, your app drops into the debugger. 189 190- If you Release() an endpoint too many times, your app *could* drop 191 into the debugger. It might also crash, because you are now using a 192 dead object. It depends on whether the memory that was previously 193 occupied by your endpoint object was overwritten in the mean time. 194 195- You are allowed to pass NULL into the constructors of 196 BMidiLocalConsumer and BMidiLocalProducer, in which case the 197 endpoint's name is simply an empty string. 198 199Changing endpoint attributes 200~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 201 202- An endpoint can be "invalid". In the case of a proxy this means that 203 the remote endpoint is unregistered or even deleted. Local endpoints 204 can only be invalid if something went wrong during their creation (no 205 connection to server, for example). You can get the attributes of 206 invalid objects, but you cannot set them. Any attempts to do so will 207 return an error code. 208 209- For changing the name, latency, or properties of an endpoint, 210 libmidi2 sends an 'Mchg' message with the fields that should be 211 changed, "midi:name", "midi:latency", or "midi:properties". 212 Registering or unregistering an endpoint also sends such an 'Mchg' 213 message, because we consider the "registered" state also an 214 attribute, in "midi:registered". The message obviously also includes 215 the ID of the endpoint in question. Properties are sent using a 216 different message, because the properties are not stored inside the 217 BMidiEndpoints. 218 219- After handling the 'Mchg' request, the midi_server broadcasts an 220 'mCHG' notification to all the other apps. This message has the same 221 contents as the original request. 222 223- If the 'Mchg' message contains an invalid "midi:id" (i.e. no such 224 endpoint exists or it does not belong to the app that sent the 225 request), the midi_server returns an error code, and it does not 226 notify the other apps. 227 228- If you try to Register() an endpoint that is already registered, 229 libmidi2 does not send a message to the midi_server but simply 230 returns B_OK. (Be's implementation *did* send a message, but our 231 libmidi2 also keeps track whether an endpoint is registered or not.) 232 Although registering an endpoint more than once doesn't make much 233 sense, it is not considered an error. Likewise for Unregister()ing an 234 endpoint that is not registered. 235 236- If you try to Register() or Unregister() a remote endpoint, libmidi2 237 immediately returns an error code, and does not send a message to the 238 server. Likewise for a local endpoints that are invalid (i.e. whose 239 IsValid() function returns false). 240 241- BMidiRoster::Register() and Unregister() do the same thing as 242 BMidiEndpoint::Register() and Unregister(). If you pass NULL into 243 these functions, they return B_BAD_VALUE. 244 245- SetName() ignores NULL names. When you call it on a remote endpoint, 246 SetName() does nothing. SetName() does not send a message if the new 247 name is the same as the current name. 248 249- SetLatency() ignores negative values. SetLatency() does not send a 250 message if the new latency is the same as the current latency. (Since 251 SetLatency() lives in BMidiLocalConsumer, you can never use it on 252 remote endpoints.) 253 254- We store a copy of the endpoint properties in each BMidiEndpoint. The 255 properties of new endpoints are empty. GetProperties() copies this 256 BMessage into the client's BMessage. GetProperties() returns NULL if 257 the message parameter is NULL. 258 259- SetProperties() returns NULL if the message parameter is NULL. It 260 returns an error code if the endpoint is remote or invalid. 261 SetProperties() does *not* compare the contents of the new BMessage 262 to the old, so it will always send out the change request. 263 264Connections 265~~~~~~~~~~~ 266 267- BMidiProducer::Connect() sends an 'Mcon' request to the midi_server. 268 This request contains the IDs of the producer and the consumer you 269 want to connect. The server sends back a reply with a result code. If 270 it is possible to make this connection, the server broadcasts an 271 'mCON' notification to all other apps. In one of these apps the 272 producer is local, so that app's libmidi2 calls the 273 BMidiLocalProducer::Connected() hook. 274 275- You are not allowed to connect the same producer and consumer more 276 than once. The midi_server checks for this. It also returns an error 277 code if you try to disconnect two endpoints that were not connected. 278 279- Disconnect() sends an 'Mdis' request to the server, which contains 280 the IDs of the producer and consumer that you want to disconnect. The 281 server replies with a result code. If the connection could be broken, 282 it also sends an 'mDIS' notification to the other apps. libmidi2 283 calls the local producer's BMidiLocalProducer::Disconnected() hook. 284 285- Connect() and Disconnect() immediately return an error code if you 286 pass a NULL argument, or if the producer or consumer is invalid. 287 288- When you Release() a local consumer that is connected, all apps will 289 go through their producers, and throw away this consumer from their 290 connection lists. If one of these producers is local, we call its 291 Disconnected() hook. If you release a local producer, this is not 292 necessary. 293 294Watching 295~~~~~~~~ 296 297- When you call StartWatching(), the BMidiRosterLooper remembers the 298 BMessenger, and sends it B_MIDI_EVENT notifications for all 299 registered remote endpoints, and the current connections between 300 them. It does not let you know about local endpoints. When you call 301 StartWatching() a second time with the same BMessenger, you'll 302 receive the whole bunch of notifications again. StartWatching(NULL) 303 is not allowed, and will be ignored (so it is not the same as 304 StopWatching()). 305 306Thread safety 307~~~~~~~~~~~~~ 308 309- Within libmidi2 there are several possible race conditions, because 310 we are dealing with two threads: the one from BMidiRosterLooper and a 311 thread from the client app, most likely the BApplication's main 312 thread. Both can access the same data: BMidiEndpoint objects. To 313 synchronize these threads, we lock the BMidiRosterLooper, which is a 314 normal BLooper. Anything happening in BMidiRosterLooper's message 315 handlers is safe, because BLoopers are automatically locked when 316 handling a message. Any other operations (which run from a different 317 thread) must first lock the looper if they access the list of 318 endpoints or certain BMidiEndpoint attributes (name, properties, 319 etc). 320 321- What if you obtain a BMidiEndpoint object from FindEndpoint() and at 322 the same time the BMidiRosterLooper receives an 'mDEL' request to 323 delete that endpoint? FindEndpoint() locks the looper, and bumps the 324 endpoint object before giving it to you. Now the looper sees that the 325 endpoint's refcount is larger than 0, so it won't delete it (although 326 it will remove the endpoint from its internal list). What if you 327 Acquire() or Release() a remote endpoint while it is being deleted by 328 the looper? That also won't happen, because if you have a pointer to 329 that endpoint, its refcount is at least 1 and the looper won't delete 330 it. 331 332- It is not safe to use a BMidiEndpoint and/or the BMidiRoster from 333 more than one client thread at a time; if you want to do that, you 334 should synchronize access to these objects yourself. The only 335 exception is the Spray() functions from BMidiLocalProducer, since 336 most producers have a separate thread to spray their MIDI events. 337 This is fine, as long as that thread isn't used for anything else, 338 and it is the only one that does the spraying. 339 340- BMidiProducer objects keep a list of consumers they are connected to. 341 This list can be accessed by several threads at a time: the client's 342 thread, the BMidiRosterLooper thread, and possibly a separate thread 343 that is spraying MIDI events. We could have locked the producer using 344 BMidiRosterLooper's lock, but that would freeze everything else while 345 the producer is spraying events. Conversely, it would freeze all 346 producers while the looper is talking to the midi_server. To lock 347 with a finer granularity, each BMidiProducer has its own BLocker, 348 which is used only to lock the list of connected consumers. 349 350Misc remarks 351~~~~~~~~~~~~ 352 353- BMidiEndpoint keeps track of its local/remote state with an "isLocal" 354 variable, and whether it is a producer/consumer with "isConsumer". It 355 also has an "isRegistered" field to remember whether this endpoint is 356 registered or not. Why not lump all these different states together 357 into one "flags" bitmask? The reason is that isLocal only makes sense 358 to this application, not to others. Also, the values of isLocal and 359 isConsumer never change, but isRegistered does. It made more sense 360 (and clearer code) to separate them out. Finally, isRegistered does 361 not need to be protected by a lock, even though it can be accessed by 362 multiple threads at a time. Reading and writing a bool is atomic, so 363 this can't get messed up. 364 365The messages 366~~~~~~~~~~~~ 367 368 :: 369 370 Message: Mapp (MSG_REGISTER_APP) 371 BMessenger midi:messenger 372 Reply: 373 (no reply) 374 375 Message: mAPP (MSG_APP_REGISTERED) 376 (no fields) 377 378 Message: Mnew (MSG_CREATE_ENDPOINT) 379 bool midi:consumer 380 bool midi:registered 381 char[] midi:name 382 BMessage midi:properties 383 int32 midi:port (consumer only) 384 int64 midi:latency (consumer only) 385 Reply: 386 int32 midi:result 387 int32 midi:id 388 389 Message: mNEW (MSG_ENPOINT_CREATED) 390 int32 midi:id 391 bool midi:consumer 392 bool midi:registered 393 char[] midi:name 394 BMessage midi:properties 395 int32 midi:port (consumer only) 396 int64 midi:latency (consumer only) 397 398 Message: Mdel (MSG_DELETE_ENDPOINT) 399 int32 midi:id 400 Reply: 401 (no reply) 402 403 Message: Mdie (MSG_PURGE_ENDPOINT) 404 int32 midi:id 405 Reply: 406 (no reply) 407 408 Message: mDEL (MSG_ENDPOINT_DELETED) 409 int32 midi:id 410 411 Message: Mchg (MSG_CHANGE_ENDPOINT) 412 int32 midi:id 413 int32 midi:registered (optional) 414 char[] midi:name (optional) 415 int64 midi:latency (optional) 416 BMessage midi:properties (optional) 417 Reply: 418 int32 midi:result 419 420 Message: mCHG (MSG_ENDPOINT_CHANGED) 421 int32 midi:id 422 int32 midi:registered (optional) 423 char[] midi:name (optional) 424 int64 midi:latency (optional) 425 BMessage midi:properties (optional) 426 427-------------- 428 429MIDI events 430----------- 431 432- MIDI events are always sent from a BMidiLocalProducer to a 433 BMidiLocalConsumer. Proxy endpoint objects have nothing to do with 434 this. During its construction, the local consumer creates a kernel 435 port. The ID of this port is published, so everyone knows what it is. 436 When a producer sprays an event, it creates a message that it sends 437 to the ports of all connected consumers. 438 439- This means that the Midi Kit considers MIDI messages as discrete 440 events. Hardware drivers chop the stream of incoming MIDI data into 441 separate events that they send out to one or more kernel ports. 442 Consumers never have to worry about parsing a stream of MIDI data, 443 just about handling a bunch of separate events. 444 445- Each BMidiLocalConsumer has a (realtime priority) thread associated 446 with it that waits for data to arrive at the port. As soon as a new 447 MIDI message comes in, the thread examines it and feeds it to the 448 Data() hook. The Data() hook ignores the message if the "atomic" flag 449 is false, or passes it on to one of the other hook functions 450 otherwise. Incoming messages are also ignored if their contents are 451 not valid; for example, if they have too few or too many bytes for a 452 certain type of MIDI event. 453 454- Unlike the consumer, BMidiLocalProducer has no thread of its own. As 455 a result, spraying MIDI events always happens in the thread of the 456 caller. Because the consumer port's queue is only 1 message deep, 457 spray functions will block if the consumer thread is already busy 458 handling another MIDI event. (For this reason, the Midi Kit does not 459 support interleaving of real time messages with lower priority 460 messages such as sysex dumps, except at the driver level.) 461 462- The producer does not just send MIDI event data to the consumer, it 463 also sends a 20-byte header describing the event. The total message 464 looks like this: 465 466 +---------+------------------------------+ 467 | 4 bytes | ID of the producer | 468 +---------+------------------------------+ 469 | 4 bytes | ID of the consumer | 470 +---------+------------------------------+ 471 | 8 bytes | performance time | 472 +---------+------------------------------+ 473 | 1 byte | atomic (1 = true, 0 = false) | 474 +---------+------------------------------+ 475 | 3 bytes | padding (0) | 476 +---------+------------------------------+ 477 | x bytes | MIDI event data | 478 +---------+------------------------------+ 479 480- In the case of a sysex event, the SystemExclusive() hook is only 481 called if the first byte of the message is 0xF0. The sysex end marker 482 (0xF7) is optional; only if the last byte is 0xF7 we strip it off. 483 This is unlike Be's implementation, which all always strips the last 484 byte even when it is not 0xF7. According to the MIDI spec, 0xF7 is 485 not really required; any non-realtime status byte ends a sysex 486 message. 487 488- SprayTempoChange() sends 0xFF5103tttttt, where tttttt is 489 60,000,000/bpm. This feature is not really part of the MIDI spec, but 490 an extension from the SMF (Standard MIDI File) format. Of course, the 491 TempoChange() hook is called in response to this message. 492 493- The MIDI spec allows for a number of shortcuts. A Note On event with 494 velocity 0 is supposed to be interpreted as a Note Off, for example. 495 The Midi Kit does not concern itself with these shortcuts. In this 496 case, it still calls the NoteOn() hook with a velocity parameter of 497 0. 498 499- The purpose of BMidiLocalConsumer's AllNotesOff() function is not 500 entirely clear. All Notes Off is a so-called "channel mode message" 501 and is generated by doing a SprayControlChange(channel, 502 B_ALL_NOTES_OFF, 0). BMidi has an AllNotesOff() function that sends 503 an All Notes Off event to all channels, and possible Note Off events 504 to all keys on all channels as well. I suspect someone at Be was 505 confused by AllNotesOff() being declared "virtual", and thought it 506 was a hook function. Only that would explain it being in 507 BMidiLocalConsumer as opposed to BMidiLocalProducer, where it would 508 have made sense. The disassembly for Be's libmidi2.so shows that 509 AllNotesOff() is empty, so to cut a long story short, our 510 AllNotesOff() simply does nothing and is never invoked either. 511 512- There are several types of System Common events, each of which takes 513 a different number of data bytes (0, 1, or 2). But 514 SpraySystemCommon() and the SystemCommon() hook are always given 2 515 data parameters. The Midi Kit simply ignores the extra data bytes; in 516 fact, in our implementation it doesn't even send them. (The Be 517 implementation always sends 2 data bytes, but that will confuse the 518 Midi Kit if the client does a SprayData() of a common event instead. 519 In our case, that will still invoke the SystemCommon() hook, because 520 we are not as easily fooled.) 521 522- Handling of timeouts is fairly straightforward. When reading from the 523 port, we specify an absolute timeout. When the port function returns 524 with a B_TIMED_OUT error code, we call the Timeout() hook. Then we 525 reset the timeout value to -1, which means that timeouts are disabled 526 (until the client calls SetTimeout() again). This design means that a 527 call to SetTimeout() only takes effect the next time we read from the 528 port, i.e. after at least one new MIDI event is received (or the 529 previous timeout is triggered). Even though BMidiLocalConsumer's 530 timeout and timeoutData values are accessed by two different threads, 531 I did not bother to protect this. Both values are int32's and 532 reading/writing them should be an atomic operation on most processors 533 anyway. 534 535.. |image0| image:: midi_server.png 536.. |image1| image:: libmidi2.png 537 538
