The call that started it
At 9:17 on a Thursday in March 2024, 36 hours before a field trial, my phone rang. A client had 60 LoRaWAN sensor nodes on a trailer, ready to install for a water-management demo. Every node was built around the Semtech SX1276 LoRa transceiver, and the gateway backhaul was an XR60 5G/LTE router. The demo was supposed to run for three days in front of an investor group. At 8:00 that morning, 14 of the 60 nodes had fallen off the network map.
The first thing I asked wasn't 'Why are they failing?' It was 'How much time do we have?' That's how I triage every emergency. Time determines everything.
The wrong assumption
I should say this: Semtech, the company behind LoRa and LoRaWAN, wasn't the problem. The SX1276 is a tough little chip. But that's precisely why people usually make the wrong assumption.
The assumption is that a node dropping off a LoRaWAN network means a radio problem. The reality is that power delivery failures often look exactly like radio problems. A capacitor with no capacitance left can cause the transceiver to brown out under load. The chip restarts, the node disappears, and by the time someone reconnects it, the network map shows a gap. According to the SX1276 datasheet, that radio can reach -137 dBm sensitivity in LoRa mode. It can hear a whisper. It can't hear anything if it's rebooting because a power rail is collapsing.
Stalking the usual suspects
We checked the obvious things first. Antenna connections? Tight. The IPEX connector on the module? Seated. The external SMA connector? No corrosion, no bent pin. All 60 nodes had the same Semtech LoRaWAN configuration. The gateway was listening. The XR60 router had a solid 5G uplink.
Then I asked the production manager what had changed. He said: 'Nothing. Same boards we've used for a year.'
That's when I stopped trusting the network and opened the last failing node.
Why I grabbed the multimeter
There are two ways to handle a weird field failure: swap the board and keep going, or stop and measure. I went back and forth for a few minutes. Swapping would've been faster, and we had spare boards. But if we deployed the same bad design again, we'd be back out there next week. So I grabbed a multimeter and shut the door.
If you want to know how to test a capacitor with a multimeter, here's the method I used. It's not magic, but it'll save you a lot of pain:
- Disconnect power. Then discharge the capacitor with a resistor. I use a 1kΩ resistor, not a screwdriver, because a screwdriver can crack a trace.
- Switch the multimeter to capacitance mode. It's usually marked with the μF symbol.
- Touch the probes to the capacitor leads and wait for the reading to stabilize. It may take a second or two.
- Compare the reading to the number printed on the side. A healthy capacitor should read within its tolerance, usually ±20%. A bad one reads low, zero, or open.
The first bad node had a 10 μF capacitor on its 3.3 V rail. It read 0.3 μF. That wasn't a slight drift. It was dead.
What the multimeter told us
That reading changed everything. We cracked open the other 13 nodes that had dropped offline, and eight of them had the same capacitor on the same rail. It was a bad batch, not random failures.
And this is the part that still gets me: if I'd checked one node when they arrived, we'd have known twelve hours earlier. Five minutes of verification beats five days of correction.
Semtech technologies are solid, but a radio link is only as good as the board around it. The SX1276 can send a packet through a fence if the power rail is clean. It can't do anything if the chip shuts down because a capacitor is missing half its capacitance.
By the way, the Semtech company documentation is blunt about this. The reference designs list decoupling capacitors with specific values and placement. I'd read it. I just hadn't followed it with enough discipline.
The fix and the buffer
Replacing 14 capacitors sounds simple, but it wasn't. We had to source the right caps from a local distributor, pay $80 extra in expedited shipping, reseat every U.FL connector, and run a 45-minute soak test on all 60 nodes. The budget hit was real. The alternative was worse: missing the field trial would have triggered a $30,000 penalty clause in the client's contract.
At midnight, three of us were still in the shop. The client wanted to rush out at 4:00 AM. I said no. We needed the full 45-minute test to catch any other weak capacitors. That's the decision I'm most proud of, even though it made everyone tired.
The demo went live at 8:00 AM as scheduled. It ran for three days without one lost packet.
What I'd do differently
Looking back, I should have bench-tested every node before we loaded the trailer. At the time, the schedule felt too tight. It turned out to be much tighter after we were in the field.
I still kick myself for not asking the board vendor for capacitor test data before we placed the order. If I'd done that, we'd have known about the bad batch before it cost us a night's sleep.
The checklist we now use
Since that incident, I've built the capacitor and connector check into every deployment. The 12-point checklist I created after this mistake has saved us an estimated $8,000 in potential rework. It goes like this:
- Bench-test every board before it leaves the shop. Test at least one capacitor per power rail.
- Check every RF connector—U.FL, SMA, or RP-SMA—with a continuity test.
- Inspect the board for discolored capacitors before installation.
- If you replace a board, test it again. Don't trust new boards just because they're new.
Maybe that sounds obsessive. But I learned the hard way: prevention is cheaper than panic. And when you're standing in a field with 60 nodes and investors arriving in the morning, panic is the most expensive technology you can use.
