readBlockData() sometimes times out after 200ms

[RonW144]

We have 7 analog input modules on our P1AM-100 system:
Power supply: P1-01AC on 120VAC
Shield: P1AM-SERIAL with nothing connected.
CPU: P1AM-100 with USB serial to a PC
Module 1: P1-04AD
Module 2: P1-04RTD
Modules 3-7: P104THM
Module 8: P1-08TRS (Relay outputs)

The following code usually works, but occasionally times out after 200ms. It does this even if we have nothing attached to any modules.

char buffer[112];
P1.readBlockData(buffer,112,0,ANALOG_IN_BLOCK);

This is with Arduino IDE 2.3.5 and P1AM 1.0.9.

[AdamCummick]

Is your loop exclusively ?

whilte(1){
  char buffer[112];
  P1.readBlockData(buffer,112,0,ANALOG_IN_BLOCK);
}

[RonW144]

No, it's part of a much larger program. I simplified it to make the point. Here's the actual code:

const int ANALOGINPUTBLOCKLEN = 4*4*7; // 4 bytes/channel, 4 channels/module, 7 modules.
char analogInputData[ANALOGINPUTBLOCKLEN]; // Data for block read of analog modules.

// Read data as a block.
unsigned long duration_ms = millis(); // We want to see how long the call to readBlockData() takes.
P1.readBlockData(analogInputData,ANALOGINPUTBLOCKLEN,0,ANALOG_IN_BLOCK);
duration_ms = millis() - duration_ms; // Duration of readBlockData().

// See if the call to readBlockData() took too long.
if (50 < duration_ms)
{
// Print debug message if the failure just happened, not if it's ongoing.
if (!analogFailure)
{
Serial.print("99:readBlockData took "); Serial.print(duration_ms); Serial.println(" ms;");

// The call to readBlockData() took too long. P1 modules must be reconfigured.
analogFailure = true;
}
}

This usually works, but occasionally prints "99:readBlockData took 200 ms;"
analogFailure is a flag that causes another part of the program to reset the modules and clear analogFailure.

[AdamCummick]

I'll see if we can reproduce this. Is it always exactly 200ms?

[RonW144]

No. I've seen 203ms and 206ms.

[RonW144]

I forgot to mention that I tried breaking down the single readBlockData() call into multiple calls, each accessing one board or one type of board. For example, in the code above I replaced the single readBlockData() call with this:
P1.readBlockData(analogInputData,16,0,ANALOG_IN_BLOCK);
P1.readBlockData(analogInputData + 16,16,16,ANALOG_IN_BLOCK);
P1.readBlockData(analogInputData + 32,16,32,ANALOG_IN_BLOCK);
P1.readBlockData(analogInputData + 48,16,48,ANALOG_IN_BLOCK);
P1.readBlockData(analogInputData + 64,16,64,ANALOG_IN_BLOCK);
P1.readBlockData(analogInputData + 80,16,80,ANALOG_IN_BLOCK);
P1.readBlockData(analogInputData + 96,16,96,ANALOG_IN_BLOCK);
Each call accesses a single board. I still got the error. That kind of experiment may be where the 203ms and 206ms durations came from.

[AdamCummick]

Hey, so I was running an overnight test on this to reproduce, but my power supply shut off so no results. I'll re-run today and try and see this error.

Do you have any power instability on your system? Any electrically noisy components nearby?

[RonW144]

I'm running this in my office cube with nothing attached to the analog inputs, so stable power and nothing noisy nearby.
In my experiment where I broke it down by board (see my previous post) I timed each call, and so far it's only the first call (which reads the P1-04AD) that occasionally takes 200ms. All the others are no more than 2ms, usually 1ms, sometimes 0, according to millis().

[RonW144]

I updated my one-readBlockData-per-card code to comment out the call to the first card (P1-04AD). Now the failure only occurs with the second card (P1-04RTD) which is now the first one I try to access.
It seems like the issue is more likely to occur if I set outputs on the P1-08RTS. That's the last card. I just use P1.writeDiscrete() to access it. I don't have anything connected to those outputs, so all that happens is the indicator lights turn on.

[AdamCummick]

I ran your original code over the weekend and did not encounter this issue with the first output constantly turned on. What outputs did you have on here.

Is this all newly purchased hardware or has it been in the field or heavily used?

[RonW144]

It's new, with the exception of the relay unit (P1-08RTS). We burned out channel 5 on that unit and are using channel 6 instead. We are using P1.writeDiscrete() for the relay unit, which does not call spiTimeout(), which is what generates the error.
I've been running my system for about an hour, and had five errors. Most of them occurred toward the end of the hour.

[RonW144]

I just realized I didn't answer your question about which outputs we use. We're using NO1, NO2, NO4, and NO6.

[AdamCummick]

If you remove the P1-08TRS from the system, do the errors persist?

[RonW144]

I don't know, but since the errors seem to occur when we've asserted an output, if we remove the P1-08TRS and the errors go away, that doesn't really prove anything. I'll see if I can create a test project that does nothing but access the analog modules.

Do you know if there's anywhere I can get the pinout of the 20-pin buss that goes through all the modules? I'm thinking if there's a tenuous ground through the modules, or an SPI termination issue, we could address it if we knew the pinout.

[AdamCummick]

Just looking for the root cause as I wouldn't expect a 08TRS to cause issues as it's one of our most often used modules in testing.

Unfortunately, I cannot provide any information on the module bus.