ECU Diagnostics – part 14 : Software Framework mbe.py

This is hopefully a reasonably short post about the software framework I’ve developed to help read sensor data from the MBE 9A4 ECU.

The software is written in Python 3 and is available here: Caterham-OBD., however, I’m going to talk about using one file only which is mbe.py

The idea behind mbe.py is to “abstract” the setup and processing of requests and responses to an MBE 9A4 ECU and to provide a “human readable” set of results. The code implements the MBE-ISOTP protocol and can create multiple compound CAN bus requests and process multiple compound CAN bus responses. 

The code is implemented as a Python class with a few member functions to initialise the class and then send and process messages to the ECU.

There’s an example test program here.

The Programming Interface (API) to the code is as follows: 

mbe()

Creates an mbe class object with no parameters

set_options(vars_file, request_id, response_id, interface)

Sets the mbe options.

  • vars_file: A JSON encoded representation of the Easimap EC2 file. This is used to define all the variables the ECU can process. It also sets offset and scaling configurations for each of the variables. This file is created using ec2parse.py and the latest version of the JSON vars_file can be found at 9A4be52a.ec2.utf8.json.
  • request_id: This is the CAN bus ID needed to tell mbe.py what ID to use when making requests to the ECU. This parameter may be ignored and mbe.py will use the default 0x0cbe1101.
  • response_id: This is the CAN bus ID needed to tell mbe.py what ID to look for when receiving responses for the ECU. This parameter may be ignored and mbe.py will use the default 0x0cbe0111.
  • interface: defaults to can0
  • RETURN: Returns False is there was a problem setting these options

add_variable_list(list)

Adds a list of variables to the mbe class to be processed later.

  • list: an array of strings defining which ECU variables to request and process with process_all_pages()
  • RETURN: False if there is problem with the list, otherwise returns True

bind()

Opens and binds mbe.py to can-isotp kernel module.

  • RETURN: True

process_all_pages(results)

Processes all variables.

  • results: A dictionary indexed by the variable name string and supplying results, units, and human readable string for each variable.
  • RETURN: a dictionary of results, otherwise returns False if there’s a problem.

A simple example might be as follows:

ecu = mbe.mbe()
ecu.set_options("9A4be52a.ec2.utf8.json", 0x0cbe1101, 0x0cbe0111, "can0")
ecu.add_variable_list_to_follow([RT_ENGINESPEED])
ecu.bind()
ecu.process_all_pages(results)

 

ECU Diagnostics – part 13.3 : MBE-ISOTP

SUMMARY

We now understand the communications protocol used by Easimap and can use it to ask the ECU for whatever data the car has to offer. We can then manipulate the responses we get back from the car and format them to make sense to us.

TL;DR

I’ve got to a point now where I think we understand how Easimap is talking to the car and how it formats the data fields it gets from the car. And therefore we can just do the same to get whatever data the car has to offer.

What I haven’t looked at yet is how Easimap can set parameters in the car’s ECU. I will take a look at that at some point but for the moment my interest is in creating some sort of Linux tool that can extract data from the car and display it. I may also have a look at whether the mappings on the ECU can be changed, though the general consensus is that the ECU’s are locked, and I have no reason to change that theory at the moment. I’m not sure how that locking occurs or whether it can be circumvented. At the moment I’m not interested in that side of things and have no intention of looking at it – at least at the moment.

So, to that end… here’s what I understand now about what I call the “MBE-ISOTP” protocol used by MBE in their Easimap software and the ECU’s.

Firstly, this protocol is a multi-layered protocol. It’s a bit like Russian Dolls, the actual data packets used by MBE are a couple of layers deep inside the doll. In order to look at the data the way that the ECU does we have to capture the data from the OBD port CAN bus and then unpack it a couple of times to get at the real data.

The unpacking layers in this case are CAN, ISOTP (ISO-15765) and a proprietary MBE layer.

Lets dig into the various layers and explain what’s going on, hopefully this will help future developers.

CAN LAYER

Here’s a refresher on the CAN bus layer…

The OBD-II port (on my car (2017 420R) and most modern-ish Caterhams) connects to the ECU by way of a two wire differential voltage protocol. It uses CAN-H and CAN-L to signal 1s and 0s on the CAN bus.

There are two critical things here in each CAN packet… firstly there’s the CAN ID (identifier) and then there’s the Data. The CAN ID can be either 11bits long or 29bits long. On my car with the MBE 9A4 ECU, it uses 29bit IDs. But other Caterhams also seem to use 11bit, probably with unlocked ECUs.

Secondly, the CAN packet contains up to 8 bytes of data. The CAN standard does allow for more than 8 bytes but it doesn’t seem as though our cars don’t use that version of the protocol.

Other than the ID and data, the CAN packet also contains lots of start bits, CRC (cyclic redundancy check) error checking, data length parameters and an ACK section. The ACK section is perhaps worth a few words: a sending device on the network only knows if “something” has received it’s CAN packet if a receiver sets the ACK bit when it received the data.

Anyway, on to the more interesting stuff.

The 29bit CAN IDs are used as payload type identifier – When a device sending on the bus sets a particular ID, it is telling any receiver what sort of protocol to expect in the packet.  I’ll talk more about the specifics of the IDs used in our cars when I get onto the ISOTP layer below.

So, at a high level view of each CAN packet, there are the two things we’re interested in, the CAN ID and the Data. For our cars the IDs are 29bit and the data can be up to 8 bytes.

And a packet looks like this when we’ve captured it from the CAN bus:

#   Time        Prot Len Info CAN ID       Data
168 1.612770860 CAN  32  XTD: 0x0cbe1101   10 08 01 00 00 00 00 f8
169 1.613462515 CAN  32  XTD: 0x0cbe1101   21 7c 7d 00 00 00 00 00
170 1.614548554 CAN  32  XTD: 0x0cbe0111   03 81 34 12 00 00 00 00

This data is laid out in columns by the packet sniffing tool I’m using (Wireshark/tshark). 

  • # = Frame Number
  • Time = number of seconds since the packet capture started
  • Prot = Protocol (CAN)
  • Len  = Length of the capture packet. This is not the CAN data length, that’s always 8 for us
  • CAN ID = 29bits (shown as hexadecimal in this example)
  • Data = Data in the CAN frame, shown as hexadecimal bytes

You’ll see why I picked those particular frames a bit later.

From the three packets above we can see that in order to talk to the ECU using this protocol you need to set any request packets with a CAN ID of 0x0cbe1101
and expect a response from 0x0cbe0111.

So, to recap… when we put a packet sniffer (Raspberry Pi with PICAN2 interface in my case) and watch (sniff) the CAN bus, we see a succession of lines of output from our packet sniffer like above.

ISOTP

Ok. So lets look at one level of Russian Dolls down (smaller doll).

As you can see from the example above, our particular implementation of the CAN protocol allows the car and any other device on the CAN bus to talk in chunks of 8 bytes at a time.

Now, this is quite restrictive for a modern car. The communication between Easimap and the car ECU might need to send all sorts of data that is longer than 8 bytes. For instance a 2 dimensional engine fueling map that might be hundreds of bytes long.

The MBE ECU and Easimap get over this problem by adding another protocol on top of the CAN bus interface (another Russian Doll). This protocol is called ISO-15765 or ISO-TP. By formatting the data in the CAN bus frames we can send one ISO-TP “packet” across multiple CAN bus frames – allowing us to send more than 8 bytes in a frame.

Here’s the same 3 CAN packets from above, but this time we looking at it through an ISO-TP dissector (its dissecting the packet stream and viewing it as though it is formatted in accordance with the ISO-TP protocol).

#   Time        Proto    Len Info                     Data
168 1.6127708 ISO15765 32 First Frame(Frame Len: 8) 01 00 00 00 00 f8
169 1.6134625 ISO15765 32 Consecutive Frame(Seq: 1) 7c 7d
170 1.6145485 ISO15765 32 Single Frame(Len: 3)      81 34 12

Packets 168 and 169 are actually a single ISO-TP frame. The “First Frame” and the “Consecutive Frame” are “fragments” of a whole frame and can be joined together to form an ISO-TP frame with the following data:

  01 00 00 00 00 f8 7c 7d

The “Frame Len: 8” in the First Frame tells us there are 8 bytes of data at the ISO-TP protocol layer.

Now, you might say… but hang on, we could send 8 bytes of data with just CAN frames. Well, that’s true in this particular example but for more complex examples we can send over 4000 bytes of data in a single ISO-TP packet (using many CAN frames).

One significant departure that MBE seem to make from the ISO-TP standard is that they do not implement flow-control in the protocol. We don’t need to go into that here, but if anyone is trying to talk to an MBE ECU using their own software, then they’ll need to remove any notion of flow control from the ISO-TP layer they use.

Right, so now we’ve used up two layers of the Russian Doll. The CAN layer and the ISO-TP layer. In the third layer (I’ll call it the MBE layer) we see exactly what the ECU and Easimap are really saying to each other.

MBE LAYER

The data we extracted using the ISO-TP dissector has given us the following (same as above):

  01 00 00 00 00 f8 7c 7d

What does that mean to the ECU?

Well, this took a bit of figuring out. We don’t have MBE’s complete rule-book for decoding this layer of the communications. But what we do have is their Easimap software. That helps us in two ways. Firstly, we can watch Easimap and the ECU talking to each other. That’s how we got these sample packets. But that just looks like random data at the moment. Secondly, we have the EC2 data files that Easimap loads when it discovers what ECU its talking to. MBE supports a whole raft of ECU’s and each seems to communicate with Easimap in a different way. MBE could have coded each of those protocols into Easimap but that would mean that they needed to update the Easimap software whenever they make a new ECU for a customer (Caterham or whoever).

So, MBE wrote Easimap to discover what ECU its talking to and then load the correct EC2 file for that ECU. Fortunately for us the EC2 files are easily found in the Easimap installation.

A DIGRESSION INTO EC2 FILES

The EC2 files are worth a whole post in themselves. And we had to write scripts to decode them and give us the keys to unlock the MBE layer.

In simple terms though, the EC2 files set up a lot of “variables”. Each variable is a type of data that Easimap can extract from the ECU. The 9A4 EC2 file that Easimap uses for our cars has over 2000 variables defined. It turns out that over half of them are “disabled” but that still leaves over 900 to work with. I suspect that many of these 900 don’t actually do anything with our cars but I’m planning to go and check which are “active” and which are not.

For each variable in the EC2 file there are definitions for things like, the number of bytes of data each variable takes up in the MBE communications layer, the short and long description of the variable that Easimap uses when showing the data on the screen and crucially it also has entries for what it calls a “page” and an “address”.

This information for each variable is spread all over the EC2 file and the scripts we wrote had to find each variable and pull the respective information together to build a picture of how each variable works.

{'address': '0x237c',
 'bytes': '2',
 'display_interval': '1000.000000',
 'display_maximum': '12000.000000',
 'display_minimum': '0.000000',
 'long_desc': ' Engine Speed in rpm as measured by the crankshaft sensor. This '
              'is the primary input for the fuel and ignition maps',
 'name': 'RT_ENGINESPEED',
 'page': '0xf8',
 'scale_maximum': '65535.000000',
 'scale_minimum': '0.000000',
 'short_desc': ' Engine Speed',
 'units': 'RPM'}

[ You can read the info as address=237c etc. ]

Once we’d decoded the EC2 file we could start to try and guess on the protocol used in this MBE layer.

BACK TO THE MBE LAYER

I picked all this example data and info for a reason. I had suspected ever since looking at the EC2 files for the first time a few weeks ago that the page and address info was important. And I had one of those Eureka moments when I looked at the output of the ISO-TP layer and realized from:

  01 00 00 00 00 f8 7c 7d

The f8 looked like a page from the EC2 file (see ‘page’ : ‘0xf8’ above) and the 7c looked like part of the address. So, I went away and hand decoded a bunch of messages from the ECU and low and behold it seemed to match up. So page=f8 and address Low-Byte (the 7c bit of the address) meant RT_ENGINESPEED – the engine RPMs.

The next thing was to figure out what the rest of the packet means.

And so I wrote a bunch more code and started to test out my theories.

After the [01 00 00 00 00 f8 7c 7d] message is sent by Easimap, the ECU responds with:

  0x81 0x34 0x12

One of the ways that we saw both OBD-II and ISO-TP communicate was to send a “command” or “service request” in the first couple of bytes of the packet/frame. And for the response to have a similar format but with one or more other bits in the response byte being set.

So, it seemed to make sense that the initial 0x01 byte in the frame from Easimap was responded to by the ECU with a response byte of 0x81 (0x81 in hexadecimal is 0x01 with the top bit set). This was starting to add up and so the request [01 …. 7d] was a data request for RT_ENGINESPEED and the ECU responded with 0x3412 (after the 81).

Now I’ve connived to make the data here readable. When I was first testing I didn’t have the car running and so the ECU was responding with 81 00 00… meaning 0 RPM. But when I started up the car the responses turned into real engine rev data.

Turns out that the [0x3412] is sent in reverse order and the bytes need to be swapped, making the engine revs 0x1234, which in decimal are 4660. That’s revving the car a little highly in a test, but you can see how the data works.

ADDRESSES ARE WEIRD

One of the complications around access to the ECU seems to be some weird mapping going on between the EC2 file and the address requests going to the ECU. For instance, if we take our RT_ENGINESPEED variable then the EC2 address is 0x237c.

But what Easimap sends to the ECU is a request for two bytes, a Lowest Significant Byte (LSB) and a Most Significant Byte (MSB) or 0x7c and 0x7d in our RT_ENGINESPEED example.

Having tried this out with a few dozen different variables it seems that Easimap takes the LSB of the Easimap address (0x7c in this example) and increments (adds one) for each extra byte it wants from the ECU. 

Easimap sometimes asks for 4-byte data and so this incrementing can be seen here, for example, with RT_ENGINERUNTIME…

This is the variable info for RT_ENGINERUNTIME:

{'address': '0xcfcecdcc',
 'bytes': '4',
 'display_interval': '10000.000000',
 'display_maximum': '1193046.500000',
 'display_minimum': '0.000000',
 'long_desc': ' Engine Run Time',
 'name': 'RT_ENGINERUNTIME',
 'page': '0xe2',
 'scale_maximum': '1250999.875000',
 'scale_minimum': '0.000000',
 'short_desc': ' ERT',
 'units': 'Hours'}
RT_ENGINERUNTIME=3.9094e+04 Hours ( ERT ) [0x07ffffff=134217727, Scale:1250999.875, Div:4294967295, Offset:0.0]

But that’s a bit weird too, because the EC2 address (‘0xcfcecdcc’) is the actual series of requests sent to the ECU. I’m a little baffled at the moment about when Easimap uses the EC2 address and when it makes one up. I suspect there are rules depending on how many bytes are being asked for. But obviously the sharp whitted amongst you will have realised that 0xcfcecdcc is in the correct LSB, LSB+1, LSB+2, LSB+3 order. But I’m still not sure when I have to believe in what the EC2 file has for the address and when I have to calculate it myself… I default to doing it myself and that seems to work.

COMPOUND DATA REQUESTS

Lets talk about another twist to the tale. In order that Easimap can talk to the ECU in an as-efficient way as possible. It batches up requests to pages inside the ECU and asks for a bunch of data results in one go. A request and response will therefore look like this at the CAN layer:

46 0.353164720 CAN 32 XTD: 0x0cbe1101   10 23 01 00 00 00 00 f8
47 0.353978868 CAN 32 XTD: 0x0cbe1101   21 30 31 36 37 44 45 4c
48 0.354710862 CAN 32 XTD: 0x0cbe1101   22 4d 4e 4f 50 51 5a 5b
49 0.355482989 CAN 32 XTD: 0x0cbe1101   23 5c 5d 64 6a 6b 7c 7d
50 0.356219206 CAN 32 XTD: 0x0cbe1101   24 9e 9f a0 a1 d8 d9 da
51 0.356950718 CAN 32 XTD: 0x0cbe1101   25 db 9f a0 a1 d8 d9 da
52 0.358137040 CAN 32 XTD: 0x0cbe0111   10 1e 81 4e 39 c7 47 e0
53 0.358462988 CAN 32 XTD: 0x0cbe0111   21 54 d4 83 3a 85 4e 39
54 0.358805252 CAN 32 XTD: 0x0cbe0111   22 a8 0d a2 a0 00 b0 4f
55 0.359140831 CAN 32 XTD: 0x0cbe0111   23 00 00 7a 9c 68 08 00
56 0.359467705 CAN 32 XTD: 0x0cbe0111   24 80 00 80 a1 d8 d9 da


Which then looks like this at the ISO-15765 layer:

46 0.3531647 ISO15765 First Frame(Frame Len: 35) 01 00 00 00 00 f8
47 0.3539788 ISO15765 Consecutive Frame(Seq: 1)  30 31 36 37 44 45 4c
48 0.3547108 ISO15765 Consecutive Frame(Seq: 2)  4d 4e 4f 50 51 5a 5b
49 0.3554829 ISO15765 Consecutive Frame(Seq: 3)  5c 5d 64 6a 6b 7c 7d
50 0.3562192 ISO15765 Consecutive Frame(Seq: 4)  9e 9f a0 a1 d8 d9 da
51 0.3569507 ISO15765 Consecutive Frame(Seq: 5)  db 9f a0 a1 d8 d9 da
52 0.3581370 ISO15765 First Frame(Frame Len: 30) 81 4e 39 c7 47 e0
53 0.3584629 ISO15765 Consecutive Frame(Seq: 1)  54 d4 83 3a 85 4e 39
54 0.3588052 ISO15765 Consecutive Frame(Seq: 2)  a8 0d a2 a0 00 b0 4f
55 0.3591408 ISO15765 Consecutive Frame(Seq: 3)  00 00 7a 9c 68 08 00
56 0.3594677 ISO15765 Consecutive Frame(Seq: 4)  80 00 80 a1 d8 d9 da

Which is a request like this at the MBE layer:

01 00 00 00 00 f8 30 31 36 37 44 45 4c 4d 4e 4f 50 51 5a 5b 5c 5d 64 6a 6b 7c 7d 9e 9f a0 a1 d8 d9 da db

Followed by a response of this:

81 4e 39 c7 47 e0 54 d4 83 3a 85 4e 39 a8 0d a2 a0 00 b0 4f 00 00 7a 9c 68 08 00 80 00 80

Or once the MBE layer is decoded it’s a request for:

This is a command request for data in page: f8 ...
[{'bytes': '2', 'name': 'RT_THROTTLEANGLE1(RAW)'},
 {'bytes': '2', 'name': 'RT_AIRTEMP1(LIM)'},
 {'bytes': '2', 'name': 'RT_COOLANTTEMP1(LIM)'},
 {'bytes': '2', 'name': 'RT_COOLANTFUELFACTOR'},
 {'bytes': '2', 'name': 'RT_AIRTEMPFUELFACTOR'},
 {'bytes': '2', 'name': 'RT_THROTTLEANGLEINCREASING'},
 {'bytes': '2', 'name': 'RT_TPSFUEL+TRIMBANK1'},
 {'bytes': '2', 'name': 'RT_TPSVSSPEEDIGN+TRIM1'},
 {'bytes': '1', 'name': 'RT_THROTTLESITE1'},
 {'bytes': '2', 'name': 'RT_BAROSCALEDLIM'},
 {'bytes': '2', 'name': 'RT_ENGINESPEED'},
 {'bytes': '2', 'name': 'RT_BATTERYVOLTAGE(LIM)'},
 {'bytes': '2', 'name': 'RT_BATTERYVOLTAGECOMP'},
 {'bytes': '2', 'name': 'RT_MAPPINGPOT1LIM'},
 {'bytes': '2', 'name': 'RT_MAPPINGPOT2LIM'}]

and a response with these values:

RT_THROTTLEANGLE1(RAW)=1.1192 V ( Throttle Angle 1 (Raw) ) [0x394e=14670, Scale:5.0, Div:65535, Offset:0.0]
RT_AIRTEMP1(LIM)=14.862  ( Air Temp ) [0x47c7=18375, Scale:160.0, Div:65535, Offset:-30.0]
RT_COOLANTTEMP1(LIM)=23.048  ( Coolant Temp ) [0x54e0=21728, Scale:160.0, Div:65535, Offset:-30.0]
RT_COOLANTFUELFACTOR=105.98  ( Coolant Fuel Factor ) [0x83d4=33748, Scale:400.0, Div:65535, Offset:-1e+02]
RT_AIRTEMPFUELFACTOR=4.0848  ( Air Temp Fuel Factor ) [0x853a=34106, Scale:200.0, Div:65535, Offset:-1e+02]
RT_THROTTLEANGLEINCREASING=1.1192 V ( Throttle Angle Increasing ) [0x394e=14670, Scale:5.0, Div:65535, Offset:0.0]
RT_TPSFUEL+TRIMBANK1=5.6013 ms ( TPS Fuel + Trim ) [0x0da8=3496, Scale:105.0, Div:65535, Offset:0.0]
RT_TPSVSSPEEDIGN+TRIM1=4.7022  ( TPS vs Speed Ign + Trim ) [0xa0a2=41122, Scale:-120.0, Div:65535, Offset:80.0]
RT_THROTTLESITE1=0.0 Site ( Throttle Site 1 ) [0x00=0, Scale:16.0, Div:255, Offset:0.0]
RT_BAROSCALEDLIM=1.04 Bar ( Baro Pressure ) [0x4fb0=20400, Scale:6.5535, Div:65535, Offset:-1.0]
RT_ENGINESPEED=0.0 RPM ( Engine Speed ) [0x0000=0, Scale:65535.0, Div:65535, Offset:0.0]
RT_BATTERYVOLTAGE(LIM)=12.225 V ( Battery Voltage ) [0x9c7a=40058, Scale:20.0, Div:65535, Offset:0.0]
RT_BATTERYVOLTAGECOMP=0.43116 ms ( Battery Voltage Comp ) [0x0868=2152, Scale:13.13, Div:65535, Offset:0.0]
RT_MAPPINGPOT1LIM=0.0015259  ( Mapping Pot 1 ) [0x8000=32768, Scale:200.0, Div:65535, Offset:-1e+02]
RT_MAPPINGPOT2LIM=0.0015259  ( Mapping Pot 2 ) [0x8000=32768, Scale:200.0, Div:65535, Offset:-1e+02]

All those results are what you get back from the car after sending one ISO-TP request frame and getting one response frame. That’s way more efficient than either the MBE-Broadcast or OBD-II protocols.

RESULTS

Most of the variables have scale and offset factors that have to be applied, but initial results give the following for the default Easimap screen…

RT_CURRENTFAULTSB=3.2769e+04 - ( Current Faults B ) [0x8001=32769, Scale:65535.0, Div:65535, Offset:0.0]
RT_ENGINE_CONFIG=176.0 - ( Eng Cnfg ) [0xb0=176, Scale:255.0, Div:255, Offset:0.0]
RT_MEMORYCONFIG=64.0 - ( Memory Configuration ) [0x40=64, Scale:255.0, Div:255, Offset:0.0]
RT_IGNITIONADVANCEBANK1=23.691  ( Ignition Advance ) [0x7820=30752, Scale:-120.0, Div:65535, Offset:80.0]
RT_BAROFUELCOMP=3.5294  ( Barometric Pressure Fuel Compensation ) [0x84=132, Scale:200.0, Div:255, Offset:-1e+02]
RT_CRANKCOUNT=463.0 - ( Crank Count ) [0x01cf=463, Scale:65535.0, Div:65535, Offset:0.0]
RT_ENGINERUNTIME=3.9094e+04 Hours ( ERT ) [0x07ffffff=134217727, Scale:1250999.875, Div:4294967295, Offset:0.0]
RT_THROTTLEANGLE1(RAW)=1.5767 V ( Throttle Angle 1 (Raw) ) [0x50ba=20666, Scale:5.0, Div:65535, Offset:0.0]
RT_AIRTEMP1(LIM)=14.615  ( Air Temp ) [0x4762=18274, Scale:160.0, Div:65535, Offset:-30.0]
RT_COOLANTTEMP1(LIM)=23.121  ( Coolant Temp ) [0x54fe=21758, Scale:160.0, Div:65535, Offset:-30.0]
RT_COOLANTFUELFACTOR=22.719  ( Coolant Fuel Factor ) [0x4e8a=20106, Scale:400.0, Div:65535, Offset:-1e+02]
RT_AIRTEMPFUELFACTOR=4.0848  ( Air Temp Fuel Factor ) [0x853a=34106, Scale:200.0, Div:65535, Offset:-1e+02]
RT_THROTTLEANGLEINCREASING=1.5767 V ( Throttle Angle Increasing ) [0x50ba=20666, Scale:5.0, Div:65535, Offset:0.0]
RT_TPSFUEL+TRIMBANK1=4.5887 ms ( TPS Fuel + Trim ) [0x0b30=2864, Scale:105.0, Div:65535, Offset:0.0]
RT_TPSVSSPEEDIGN+TRIM1=24.32  ( TPS vs Speed Ign + Trim ) [0x76c8=30408, Scale:-120.0, Div:65535, Offset:80.0]
RT_THROTTLESITE1=8.2824 Site ( Throttle Site 1 ) [0x84=132, Scale:16.0, Div:255, Offset:0.0]
RT_BAROSCALEDLIM=1.04 Bar ( Baro Pressure ) [0x4fb0=20400, Scale:6.5535, Div:65535, Offset:-1.0]
RT_ENGINESPEED=2400.0 RPM ( Engine Speed ) [0x0960=2400, Scale:65535.0, Div:65535, Offset:0.0]
RT_BATTERYVOLTAGE(LIM)=13.087 V ( Battery Voltage ) [0xa782=42882, Scale:20.0, Div:65535, Offset:0.0]
RT_BATTERYVOLTAGECOMP=0.33138 ms ( Battery Voltage Comp ) [0x0676=1654, Scale:13.13, Div:65535, Offset:0.0]
RT_MAPPINGPOT1LIM=0.0015259  ( Mapping Pot 1 ) [0x8000=32768, Scale:200.0, Div:65535, Offset:-1e+02]
RT_MAPPINGPOT2LIM=0.0015259  ( Mapping Pot 2 ) [0x8000=32768, Scale:200.0, Div:65535, Offset:-1e+02]
RT_SOFTCUTTIME=7800.0 RPM ( Soft Cut ) [0x1e78=7800, Scale:65535.0, Div:65535, Offset:0.0]
RT_HARDCUTTIME=7900.0 RPM ( Hard Cut ) [0x1edc=7900, Scale:65535.0, Div:65535, Offset:0.0]
RT_INJECTIONTIMEA=7.2051 ms ( Final Injection Time ) [0x1191=4497, Scale:105.0, Div:65535, Offset:0.0]
RT_DUTYCYCLEA=14.51  ( Duty Cycle ) [0x25=37, Scale:100.0, Div:255, Offset:0.0]
RT_HAVESPEEDS=169.0 - ( RT_HAVESPEEDS ) [0x00a9=169, Scale:65535.0, Div:65535, Offset:0.0]

What I don’t know

This whole discussion has unfortunately only been about a small part of what Easimap can do when talking to an MBE ECU. I’ve concentrated on the sensor data that can be extracted from the car. These data values/variables in the EC2 file all start with “RT_”, and I guess that stands for “real time” or something similar. 

There’s a whole bunch of data around the 1D, 2D and 3D maps that Easimap can in theory extract and write to the car. However, with a locked ECU I’m not sure how much of this I’ll be able to get access to. So for the moment I’m concentrating on the RT_ stuff and will come back to the rest at a later date.

That’s the MBE-ISOTP protocol. Now how do we make some use of it…

ECU Diagnostics – part 13.2 : OBD-II

The OBD-II diagnostic protocol is well defined and information about it can be found in On-board Diagnostics. It’s a mish-mash of many diffrerent standards, both physical and electrical, that I’ll refer to as OBD-II. There’s also a complete breakdown of the Services and PIDs (see below) that can be found in On-board Diagnostic PIDs.

I’ll talk a little bit about the protocol here but the main point of this post is to say what OBD-II data the Catheram MBE 9A4 ECU supports and what that looks like on the CAN bus.

OBD-II can be broken down into three main data types:

  • Services (previously called “modes”),
  • Parameter IDentifiers (PIDs) and,
  • OBD-II data

You can find all the relevant standards defined in the first link above… but many of them are behind a pay-wall. So unless you’re prepared to pay for the documents then you have to glean what you can from Google searches like I did.

Fundamentally, the bog-standard OBD-II diagnostic protocol uses a single CAN bus frame (max 8 bytes, but usually less), to request, and then get a response, for a single data value (variable) from the car’s ECU. It is a request response protocol, meaning a scanner, or active device, requests a data value and the ECU responds with the result. Both request and response take up one CAN bus frame.

The basics of the protocol request and response are as follow:

OBD-II CAN Bus Request Frame        
Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
LENGTH SERVICE ID Data 0 Data 1 Data 2 Data 3 Data 4 Data 5
OBD-II CAN Bus Response Frame        
Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
LENGTH SERVICE ID + 0x40 Data 0 Data 1 Data 2 Data 3 Data 4 Data 5

The important points here, from a 9A4 perspective, are:

  • LENGTH: First byte of request or response is the number of additional bytes in the message. Note that this is “additional” bytes, in addition to the length byte itself. So, for instance a request for engine coolant temp (PID = 0x05, byte 2) and service ID of 0x01 (byte 1) requires a length of 2, i.e. two bytes (Service ID + PID) in addition to the length byte (byte 0).
  • SERVICE ID: See OBD-II and OBD-II PIDs for a list of possible service ID’s. I’ve only verified that a 9A4 only responds to a service ID of 1 so far
  • SERIVE ID + 0x40: When responding to a request 0x40 is added to the service ID, so a request with service ID of 0x01 will get a response service ID of 0x41.
  • The ECU always responds with 8 bytes (seems to pad with 0xff)
  • 9A4 ECU responds with a CAN ID of 0x18daf101
  • Only some 15 PIDs are supported by our ECUs, see below for a list

An example request and response for coolant temp might be:

OBD-II CAN Bus Request Frame : Coolant Temp    
Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
0x02 0x01 0x05          

… and the response:

OBD-II CAN Bus Response Frame : Coolant Temp    
Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
0x03 0x41 0x05 0x3d        

We can see here that the ECU responded with a coolant temperature of 0x3d. We then need to refer to the offset and scaling algorithm found again here. For a PID of 0x05, Coolant Temp, the formula is ( Result-40 ), i.e.

Coolant Temp = 0x3d – 40 = 61 -40 = 21 C 

There’s not a lot of resolution there, just an integer. That’s one of the reason’s we might like to use either of the MBE Simple or MBE ISOTP diagnostic protocols, with the later providing a lot more detail.

PID Discovery

With all these PIDs supported by the standard and cars not having to support them all, there needs to be a way of figuring out what each car can support. This is done with PIDs 0x01, 0x20, 0x40, 0x60, 0x80, 0xx0 and 0xc0.

When a request is made to each of these PIDs, the ECU responds with a bit-field showing which of the next 32 (0x20) PIDs are supported in the 4 data bytes returned in the response, including whether the next+1 set of 32 PIDs are supported at all.

So for our ECUs we get the following communication at the start of a scanning cycle:

1672 16.267925833 0x18db33f1 02 01 00 ff ff ff ff ff
1673 16.269274940 0x18daf101 06 41 00 d8 36 80 19 ff
1724 16.776437588 0x18db33f1 02 01 20 ff ff ff ff ff
1726 16.778090875 0x18daf101 06 41 20 20 00 20 01 ff
1777 17.284954491 0x18db33f1 02 01 40 ff ff ff ff ff
1778 17.287421616 0x18daf101 06 41 40 40 10 00 00 ff

Decoding that a bit, we can see that the 9A4 ECU supports:

PID=0x00 -> 0x01-0x20: 0xd8368019
PID=0x20 -> 0x21-0x40: 0x20002001
PID=0x40 -> 0x41-0x60: 0x40100000

And if we unwrangle all the bits then we’re shown that these ECUs support the following PIDs (definitions reproduced from Wikipedia):

PID
(hex)
Data bytes returned Description Min value Max value Units Formula
00 4 PIDs supported [01 – 20]       Bit encoded
  4 Monitor status since DTCs cleared.        Bit encoded
02 2 Freeze DTC        
04 1 Calculated engine load 0 100 % (or )
05 1 Engine coolant temperature -40 215 °C
0B 1 Intake manifold absolute pressure 0 255 kPa
0C 2 Engine RPM 0 16,383.75 rpm
0E 1 Timing advance -64 63.5 ° before TDC
0F 1 Intake air temperature -40 215 °C
11 1 Throttle position 0 100 %
1C 1 OBD standards this vehicle conforms to       Bit encoded
1D 1 Oxygen sensors present (in 4 banks)       Similar to PID 13, but [A0..A7] == [B1S1, B1S2, B2S1, B2S2, B3S1, B3S2, B4S1, B4S2]
20 4 PIDs supported [21 – 40]       Bit encoded
23 2 Fuel Rail Gauge Pressure (diesel, or gasoline direct injection) 0 655,350 kPa
33 1 Absolute Barometric Pressure 0 255 kPa
40 4 PIDs supported [41 – 60]       Bit encoded
42 2 Control module voltage 0 65.535 V
4C 1 Commanded throttle actuator 0 100 %

Example OBD-II Communications

Here’s one full cycle of output that my scanner pulled from the car. Obviously, the scanner is attempting to show “real time” data and so repeatedly sends requests for each of the data values that the ECU supports, then repeats again and again to give the impression of “real time” output.

Here’s the Raw CAN bus frames first followed by what Wireshark decodes the packets as.

#    Time         CAN ID     Data
1891 18.384234117 0x18db33f1 02 01 02 ff ff ff ff ff
1892 18.386480284 0x18daf101 04 41 02 22 26 ff ff ff
1948 18.932752338 0x18db33f1 02 01 04 ff ff ff ff ff
1949 18.933708266 0x18daf101 03 41 04 00 ff ff ff ff
2004 19.480255834 0x18db33f1 02 01 05 ff ff ff ff ff
2006 19.482689349 0x18daf101 03 41 05 3d ff ff ff ff
2061 20.029324990 0x18db33f1 02 01 0b ff ff ff ff ff
2062 20.031880206 0x18daf101 03 41 0b c8 ff ff ff ff
2118 20.578419385 0x18db33f1 02 01 0c ff ff ff ff ff
2119 20.581109100 0x18daf101 04 41 0c 00 00 ff ff ff
2175 21.127303969 0x18db33f1 02 01 0e ff ff ff ff ff
2176 21.128371192 0x18daf101 03 41 0e 80 ff ff ff ff
2231 21.674841186 0x18db33f1 02 01 0f ff ff ff ff ff
2233 21.676107628 0x18daf101 03 41 0f 38 ff ff ff ff
2288 22.222260164 0x18db33f1 02 01 11 ff ff ff ff ff
2289 22.224313834 0x18daf101 03 41 11 39 ff ff ff ff
2345 22.771129266 0x18db33f1 02 01 1c ff ff ff ff ff
2346 22.773691168 0x18daf101 03 41 1c 06 ff ff ff ff
2402 23.320629505 0x18db33f1 02 01 1d ff ff ff ff ff
2403 23.323019928 0x18daf101 03 41 1d 01 ff ff ff ff
2459 23.869724288 0x18db33f1 02 01 23 ff ff ff ff ff
2460 23.872168914 0x18daf101 04 41 23 00 f0 ff ff ff
2515 24.419230119 0x18db33f1 02 01 33 ff ff ff ff ff
2517 24.421439008 0x18daf101 03 41 33 00 ff ff ff ff
2572 24.968236310 0x18db33f1 02 01 42 ff ff ff ff ff
2573 24.969567270 0x18daf101 04 41 42 2d ad ff ff ff
2629 25.516453162 0x18db33f1 02 01 4c ff ff ff ff ff
2630 25.518540406 0x18daf101 06 41 4c 00 ff ff ff ff

… and here’s how Wireshark decodes those raw CAN bus frames:

#    Time         ID         Info
1891 18.384234117 [18db33f1] Freeze DTC
1892 18.386480284 [18daf101] Freeze DTC: < 22 26 >
1948 18.932752338 [18db33f1] Calculated engine load
1949 18.933708266 [18daf101] Calculated engine load: 0.00 %
2004 19.480255834 [18db33f1] Engine coolant temperature
2006 19.482689349 [18daf101] Engine coolant temperature: 21 °C
2061 20.029324990 [18db33f1] Intake manifold absolute pressure
2062 20.031880206 [18daf101] Intake manifold absolute pressure: 200 kPa
2118 20.578419385 [18db33f1] Engine RPM
2119 20.581109100 [18daf101] Engine RPM: 0.00 rpm
2175 21.127303969 [18db33f1] Timing advance
2176 21.128371192 [18daf101] Timing advance: 0.00 °BTDC
2231 21.674841186 [18db33f1] Intake air temperature
2233 21.676107628 [18daf101] Intake air temperature: 16 °C
2288 22.222260164 [18db33f1] Throttle position
2289 22.224313834 [18daf101] Throttle position: 22.35 %
2345 22.771129266 [18db33f1] OBD standards
2346 22.773691168 [18daf101] OBD standards: EOBD
2402 23.320629505 [18db33f1] Oxygen sensors present (4 banks)
2403 23.323019928 [18daf101] Oxygen sensors present (4 banks): Bank1 sensors: 1 , Bank2 sensors: None, Bank3 sensors: None, Bank4 sensors: None
2459 23.869724288 [18db33f1] Fuel Rail Gauge Pressure
2460 23.872168914 [18daf101] Fuel Rail Gauge Pressure: 2400 kPa
2515 24.419230119 [18db33f1] Absolute Barometric Pressure
2517 24.421439008 [18daf101] Absolute Barometric Pressure: 0 kPa
2572 24.968236310 [18db33f1] Control module voltage
2573 24.969567270 [18daf101] Control module voltage: 11.693 V
2629 25.516453162 [18db33f1] Commanded throttle actuator
2630 25.518540406 [18daf101] Commanded throttle actuator: 00

So, the MBE 94A’s support the OBD-II diagnostic protocol. It’s not a huge amount of info that can be gleaned from the car, but it’s more than is available from the default broadcast MBE Simple protocol.

ECU Diagnostics – part 13.1 : MBE-Broadcast

The simplest of the three Caterham OBD port diagnostic protocols we know about is what I’ve called MBE-Broadcast.

As soon as the car is put into ignition switch position 2, the ECU starts to spit out a this stream of data on the CAN bus. Unlike the OBD-II protocol and the MBE-ISOTP protocol, this a not a request/response protocol…. the car “broadcasts” these CANbus frames all the time with no provocation from anything other than turning the car on.

The protocol is very simple and consists of individual CAN bus frames where the first byte is a message type. After the message byte are up to 7 more bytes containing the ECU data, as follows:

CAN Bus Frame        
Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
Message Data 0 Data 1 Data 2 Data 3 Data 4 Data 5 Data 6

MBE-Broadcast CAN Bus Protocol Key Information

From Simon in Birmingham, Alabama, USA (sf4018 on Blatchat)…

CAN Message Layout          
Baud Rate: 500k, Little Endian (Intel)        
CAN ID: 0x0CBB0001          
Data Frequency: 10ms, Repeat Interval 80ms
Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
4 BAT2 ? ?        
3 ?   ?        
2       ? MAP    
1 CT RPML RPMH CEL TP BAT IAT

And here’s the key to the green coloured cell from the table above.

Data List  
  Data Variable Units Scale
TP Throttle Position % 100/255
CEL Calculated Engine Load % 100/255
CT Coolant Temp C (160/255)-30
RPM Engine Speed RPM (256*H)+L
IAT Intake Air Temp C (160/255)-30
MAP Manifold Pressure kPA 122/255
BAT Battery Volts V (16/255)+2.5
Not Found:    
ITA Ignition Timing Advance Deg ?

Huge thanks to Simon for supplying this info. It had been on my list to take a look at but he knocked it out of the park with this analysis!

Simon also provided this info on his car in case it is useful for anyone else (he has a 420R Left-Hand-Drive):

MAP Sensor Technical Data (Bosch p/n 0261230044):

  • Pressure Range: 0.1-1.15 Bar (i.e. measures vacuum)
  • Voltage Limitation: 380-4700mV

MAP Scaling:

Extrapolating the technical data:

Volts

Bar

kPa

Dec

0.00

0.00

0

0

0.38

0.10

10

 

4.70

1.15

115

 

5.00

1.22

122

255

So scaling is 122/255 for kPA. Or 1.22/255 for Bar. So the scaling is correct, but doesn’t match what I was seeing from the CANbus sniffer (it was showing 200kPA/2 Bar when stopped which makes no sense).

Operation:

  • Engine Stopped: ~ 1.0 Bar
  • Engine Idling: 0.1-0.2 Bar (Throttle valve is closed, creating a vacuum).
  • Throttle Open: Pressure increases with throttle position 0.1à0.9 Bar.

Example MBE-Broadcast Communication

Here’s some example CAN bus frames that we’ll decode from a running car. Notice that they are about 10ms apart and therefore they repeat every 80ms ms.

Time         CAN ID     Data
23.622742805 0x0cbb0001 ff 00 00 00 00 00 00 00
23.632726855 0x0cbb0001 ff 00 00 00 00 00 00 00
23.642685921 0x0cbb0001 ff 44 00 00 00 00 00 00
23.652859619 0x0cbb0001 ff b3 00 00 00 00 00 00
23.662872375 0x0cbb0001 04 b3 ff 28 00 00 00 00
23.673586904 0x0cbb0001 03 27 00 0d 00 0d 00 00
23.682952338 0x0cbb0001 02 87 87 00 df 35 00 00
23.692835008 0x0cbb0001 01 56 67 06 4a 6d a9 44
23.703545091 0x0cbb0001 ff 00 00 00 00 00 00 00
23.712846871 0x0cbb0001 ff 00 00 00 00 00 00 00
23.722881278 0x0cbb0001 ff 44 00 00 00 00 00 00
23.733459089 0x0cbb0001 ff aa 00 00 00 00 00 00
23.742902975 0x0cbb0001 04 aa ff 28 00 00 00 00
23.752933955 0x0cbb0001 03 27 00 0d 00 0d 00 00
23.763345214 0x0cbb0001 02 87 87 00 bf 35 00 00
23.772954746 0x0cbb0001 01 56 63 06 4a 6e b2 44

Unfortunately we don’t know yet what the messages starting with 0xff mean, hopefully we can come back to this post and update that info later.

Lets concentrate on the ones we know something about:

23.662872375 0x0cbb0001 04 b3 ff 28 00 00 00 00 
23.673586904 0x0cbb0001 03 27 00 0d 00 0d 00 00 
23.682952338 0x0cbb0001 02 87 87 00 df 35 00 00 
23.692835008 0x0cbb0001 01 56 67 06 4a 6d a9 44

We only know anything at the moment about message types 1, 2 and 4:

Message Type 4:

Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
4 BAT2 ? ?        
Values 0xb3            

Message Type 2:

Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
2       ? MAP    
Values         0x35    

Message Type 1:

Byte 0 Byte 1 Byte 2 Byte 3 Byte 4 Byte 5 Byte 6 Byte 7
1 CT RPML RPMH TP CEL BAT IAT
Values 0x56 0x67 0x06 0x4a 0x6d 0xa9 0x44

Now feeding those values into the offset and scale table:

Data List      
  Data Variable Units Scale Value Result
TP Throttle Position % 100/255 0x4a 29%
CEL Calculated Engine Load % 100/255 0x6d 42%
CT Coolant Temp C (160/255)-30 0x56 23.9C
RPM Engine Speed RPM (256*H)+L 0x667 1639
IAT Intake Air Temp C (160/255)-30 0x44 12.67C
MAP Manifold Pressure kPA 120/255 0x35 24.9kPA
BAT Battery Volts V (16/255)+2.5 0xa9 13.1V

This compares well with the data taken in the same packet capture and requested by Easimap using the MBE ISOTP protocol:

RT_BAROFUELCOMP=3.5294 ( Barometric Pressure Fuel Compensation )
RT_CRANKCOUNT=1.414e+04 - ( Crank Count )
RT_ENGINERUNTIME=3.9094e+04 Hours ( ERT )
RT_THROTTLEANGLE1(RAW)=1.4583 V ( Throttle Angle 1 (Raw) )
RT_AIRTEMP1(LIM)=13.07 ( Air Temp )
RT_COOLANTTEMP1(LIM)=23.914 ( Coolant Temp )
RT_COOLANTFUELFACTOR=21.828 ( Coolant Fuel Factor )
RT_AIRTEMPFUELFACTOR=4.3351 ( Air Temp Fuel Factor )
RT_THROTTLEANGLEINCREASING=1.4583 V ( Throttle Angle Increasing )
RT_TPSFUEL+TRIMBANK1=3.8709 ms ( TPS Fuel + Trim )
RT_TPSVSSPEEDIGN+TRIM1=16.513 ( TPS vs Speed Ign + Trim )
RT_THROTTLESITE1=6.8392 Site ( Throttle Site 1 )
RT_BAROSCALEDLIM=1.04 Bar ( Baro Pressure )
RT_ENGINESPEED=1639.0 RPM ( Engine Speed )
RT_BATTERYVOLTAGE(LIM)=13.281 V ( Battery Voltage )
RT_BATTERYVOLTAGECOMP=0.31295 ms ( Battery Voltage Comp )
RT_MAPPINGPOT1LIM=0.0015259 ( Mapping Pot 1 )
RT_MAPPINGPOT2LIM=0.0015259 ( Mapping Pot 2 )

Battery, RPM and the temperature readings match up well. But I’m not so sure about the barometric pressure readings, that’ll take a bit more investigation.

ECU Diagnostics – part 13 : Three Diagnostic Protocols in the MBE 9A4 ECU

This one’s just a short post to introduce the next three posts. In our investigations of the MBE 9A4 ECU we’ve found three different diagnostics protocols supported on the OBD port.

The following three posts will summarise what we’ve learnt about each protocol and will be updated with new information as it becomes available.

  • 13.1 MBE-Broadcast
  • 13.2 OBD-II
  • 13.3 MBE-ISOTP