[note: copied from an email message sent on July 1st.]
Many of you will have noticed that the Jamaica station is offline. I've examined the last few weeks of data and I've formed a hypothesis about the cause of failure, and I'm also including a timeline of the events leading up to the failure. The "executive summary" is this: I believe that a failure of the Deep BIC has in turn led to a station-wide power failure.
My recommendation is that DBML should send someone immediately to remove the Deep BIC. There is a chance, perhaps small, that this will bring the station back online. Standard instrument-replacement procedures should still be followed, i.e., the Deep BIC should be retrieved to a boat before disconnecting and the cable's end should be dummy-plugged and then secured to the pylon. [We may need to send them a female-style dummy plug if they don't have any on site.]
They should also try to inspect the entire length of the Deep BIC's cable from where it exits the pylon to its end, and look for places where it may have worn through. [If the problem is the cable, which in my view is less likely, then this requires disconnection from the brain unit and eventual cable replacement, both of which require a pylon-climb. Shoe might have some suggestions about what could be done with a worn cable at the waterline if a pylon-climber isn't immediately available.]
It would also be helpful if Peter could try to connect to the station via the radio unit and see if it responds. This could tell us whether the station is still running on enough power to log its data locally or not.
What do we know about the cause? Many instruments report supply voltages (the logger, the transmitter, the BICs, the WXT), and all of them show atypical power patterns beginning in early June. This was followed by the failure of the Deep BIC, a brief return to normal power patterns, and a final, dramatic drop in power levels last Wednesday and Thursday. The station has been completely offline since then.
What is the station's current status? The implications of that final power loss are troubling: if the station can't power the datalogger, then it isn't logging its data locally either. In this case only the CTDs will continue to run, and they may run out of battery power in a few months. On the other hand, it may just be that the station cannot supply the required voltage and current to power the transmitter during its transmission cycle. In this case, some local data collection and storage might still be going on.
What is wrong with the BIC? The most likely explanation is that there has been a bulkhead failure where the connector is attached. This has happened to the underwater BICs a few times before. If there has been ocean intrusion into the BIC's case, station power/ground wires may be effectively shorted within the instrument. If so, removing the BIC and dummy-plugging its cable may revive the station because it will open the short circuit and the station can recharge its depleted batteries with the solar panels.
What about the work done last week? The failure is part of a series of events that began on June 8th, so the events of last week are at most an aggravating factor. Also, there was a station cleaning in early June but a few days went by before the trouble began so that visit is not clearly implicated either. I haven't seen a blog entry from that cleaning so I'm not sure whether they noted anything out of the ordinary about the Deep BIC at that time.
Here is the failure timeline, with all times given in local Jamaica time (UTC-5). Times are approximate, accurate to within about an hour.
Tue Jun 03, 07am : groundtruth CT connected, presumably for cleaning
Wed Jun 04, 12pm : groundtruth CT disconnected
Sun Jun 08, 08pm : Deep BIC starts reporting unusually low voltages
Mon Jun 16, 08am : Deep BIC starts missing measurements
Tue Jun 17, 08pm : Deep BIC goes offline (except for one brief revival)
Sat Jun 21, 09am : Station power cycles return to normal patterns
Wed Jun 25, 06pm : Station begins final power-level plunge
Thu Jun 26, 11am : Station offline completely
regards,
Mike J+