The Shoemaker’s Children…

….go barefoot. An old proverb that speaks to those with particular skills and aptitude often taking care of their own stuff last. And usually the case with me!

I had a couple of basement outlets / lights go intermittent, I tracked it down to a particular outlet, and so far as a bad neutral (because the voltage tester was chirping on a “dead” outlet.

This noontime, I pulled a shelf away from the wall, pulled the outlet, and heard the familiar (and unsettling) crackle-crackle of arcing as a nearby light on the circuit blinked.

The sparky-monkey who wired the basement (no doubt the former owner) used those stupid / dangerous outlet holes to make connections (called “back-stabbing”) One neutral connection was high resistance, getting warm, and thankfully broke connection before anything caught fire.

I cut out the outlet, replaced with properly screwed down and wrapped around the terminal connections, and all better.

Best of all, since the circuit in question also powered my desktop computer, I got to listen to my UPS chirp while I made the swap (NASCAR-like) and returned the circuit to service with 12 minutes to spare (my UPS is oversized for the load, so lots of battery time).

Sneaky Voltage Swell Events

Long time, no post! A pandemic will do that to you . . . suffice it to say I am healthy, relatively happy, and staying busy with both my engineering work (mostly stable through the troubles) and my yoga studio work (very different these days but busier than ever with online classes and the need to provide appropriate technology).

Today’s engineering bon-bon involves a trio of voltage swell events. Seemingly caused by a short circuit (another facility load fault, perhaps utility lighting arrestors) that causes a 1/4 cycle drop-out on one phase and resultant voltage swell following. The swell has a serious overvoltage, likely to cause problems for many types of equipment / power supplies.

This event was captured as a minor RMS voltage swell (5.1% above nominal) – but the peak voltage is very high (540 V vs 395 V normal). Note the ~30 Amp peak current swell.
This event was barely detectable as an RMS voltage swell (4.8% above nominal) – once I recognized the nature of these events, I went looking for more and came up with just this one. Almost no resultant current swell.
This is the big smoking gun event. Even though it did not register as a sag / swell event at all (the voltage drop-out being balanced by the following swell, just 3.25% over nominal) it was captured as a current swell (the highest current event over 6 weeks of data capture) – 650 Amps waveform peak. Something within the client load saw this event as a problem and drew a slug of current.

I’m just the hired gun reviewing the data, pulling out events and issues, writing a report. So I won’t be following this to a resolution or further troubleshooting on site. But this is a good example why automated report writing is not always sufficient – the Fluke 1750 analysis tools would see these events as minor voltage swells (if that). It takes a human being with some experience (I’ve reviewed over 5000 power quality data sets since 2003) to see something unusual, scratch one’s head, and dig in a bit deeper.

Pandemic Crisis: Danger / Opportunity

Things have been very interesting in the yoga world since the COVID-19 pandemic forced the studio to close in March 2020. I taught my last in person class on March 11th; the studio last class was noon on March 13th.

After a weekend of stunned silence, we were back the following Monday. We started with free classes, live-streamed via Facebook Live and archived there and on YouTube. We started simple – an iPad on a tripod, and slowly added lights, sound (headsets and ambient microphones) as we realized this was not going away quickly. Unable at that time to monetize classes, we set up a couple of membership levels for those wishing to support the studio (whose revenues went to zero almost immediately). Lots of work for the studio technologist (me) to do both in terms of physical hardware, as well as online work as we transitioned away from in person classes.

Eventually, we followed our peers, teaching virtual classes via the Zoom platform. More tech work for me – setting up a zoom station at the studio as well as at home.

My home yoga room zoom set-up (left) is also handy to watch online concerts.
The studio zoom cart (right) is perhaps the 3rd iteration of the tech – with a wifi extender, on board lights, multiple cameras and a USB microphone. We can roll it anywhere!

Once the weather warmed up, we took our show on the road, teaching 7-8 outdoor classes each week. We’ve been doing free WHY in the Parks classes for many years through the local town Parks & Rec – now we ratcheted it up, with mandatory registration (to limit capacity and facilitate contact tracing), providing for donations and paid classes. We’d get up to 100 people at some classes. Opportunities for the studio techie to set up a second battery powered speaker, to figure out a way to daisy-chain (left / right speakers for larger classes), and to manage multiple wireless headsets. Oh yeah, and haul out the trusty iPad for some time-lapse videos!

Long terms, we knew that we’d be back teaching live classes, albeit capacity limited and socially distanced, so I started to build up some technology competence. I found a low cost video switcher, and some low cost HDMI cameras (think glorified security cams) and cobbled together a four camera set up that we used first for in-studio zoom classes and eventually to capture live classes.

Left: iPad confidence / stream monitor (what the end user sees)
Center: Laptop zoom computer
Right: Preview monitor and Blackmagic ATEM Mini video switcher
AIDA HD-100A cameras with Arducam lenses – top is the wide shot, bottom is the close shot. We’ve got two other cameras off to the side. HDMI cables + 12V power run into the studio office / control area.

We started with a temporary setup to prove concept and technology (HDMI cables on the floor, cameras on tripods or clamped to carts) – eventually moving to a permanent, four camera installation. You can see the end result on the studio Vimeo page here – https://vimeo.com/westhartfordyoga

Kicking It Old School

My Christmas present to myself this year – a 1974 Gottlieb Duotron pinball machine, courtesy of a local shop that once upon a time rented, sold, and serviced pool tables, juke boxes, and pinball / arcade games, but has transitioned to a warehouse full of relics and a small business in refurbing and selling old games.

Truth be told I’ve been virtually tire-kicking games for a while now – via Facebook Marketplace and Craigslist. I knew I was serious when I took down a folding table that had become a catch-all rather than a work space, replacing it with two shelving units – leaving plenty of space in my basement office for a game. And while I was a bit concerned with dissassembly, transport, and reassembly, the shop offered delivered and set-up. Sold….

The game was mostly working, with some small issues, so I’ve spent the last few weeks cleaning, tweaking, updating. Burned out bulbs, a few functions not working properly, replacing the rubber rings / elastics and some faded plastic bits. etc. It’s all up and running!

There is a significant industry out there for information and supplies. I was able to purchase a schematic and manual from an authorized source. I bought a kit of rings / rubbers curated for this machine as well as pop bumper caps. I purchased a spares kit (blubs, fuses, a new pinball, cleaner and polisher) and some LED lamps (to judiciously replace the incandescent bulbs, especially in the back box where the lamps tend to heat / damage the backglass artwork. I even found a site that recreates the instructions and replay / award cards so familiar to those who have played.

And, as an old school engineer who has been around long enough to have played this game new, and knows my way around a schematic, but never did a lot of work with relay / ladder logic, the game is an opportunity to break out a multimeter, jumper cables, and do some serious troubleshooting.

The ingenuity and complexity of these beast is amazing – stepper units to add bonus points, count players, track first and last balls, scoring reels, a motor with cams to trigger relays periodically. The Tilt devices alone are amazing.

So yeah, Merry Christmas to me. Something for an electrical engineer who does far too little engineering or troubleshooting work these days to mess around with!

The Shoemaker’s Children Go Barefoot (no more)

For someone who makes the bulk of her income working with power quality, my own computer systems have been fairly under-protected for many years.

I picked up a stand-by UPS (APC Model ES550) many years ago (maybe 10? hard to say, might have been my second device); it has served me reasonably well. And even though I’m well aware of the nature of stand-by UPS (time delay before inverter switches on, step wave inverter output) it’s done a pretty solid job of keeping my computer up and running.

A few days ago, my home office lost power for a bit – clocks were reset, the computer switched off – and I realized it was time to upgrade the office UPS. I picked up another APC – a line interactive, sine wave output model RS 1000MS – rated for 1000VA / 600W.

It’s got plenty of juice for my needs – sitting at about 20% of load / 37 minutes of battery time with my desktop, monitor, cable modem, and a small backup server and peripheral hard drive. I’m much enamored with the front panel LED screen and the PowerChute software. And while I have not set my computer up to hibernate at the command of the UPS, that’s a possibility.

I go back a long ways – when a buck a watt was perhaps a reasonable price to pay for a small UPS. So to get all this for about $150 – well, I’m not complaining.

And I took the time to run my house cable through the internal TVSS and the Ethernet from the cable modem back to the computer through the UPS – so I’ve got a better chance of surviving nearby lightning strikes / transients – related to both transient voltages and ground potential issues. I’m not at the point of driving a ground rod and connecting an external ground though. I’m down in a basement and close to the residence service panel, so not super worried about ground issues.

And I’ve also spent some time separating critical loads (computer, monitor, cable modem, exterior drives / servers) from less critical loads (printers, speakers), plugging these latter into the TVSS only outlets. And while I was down there with the system off, I spent some time untangling the cable spaghetti, wrapping and tying off cables, neatening things up.

Saying Good-bye to an Old & Loyal Friend

Not a long, emotional post (I’ve saved those for my personal blog and for social media) but I want to note the passing of my beloved canine buddy, Elo. He lived a good long doggie life of 16 years. Some clients may have heard him in the background during phone calls – he was my shadow and wherever I was in the house / office, he followed.

Elo came into my life way back in 2004; he moved with me to my present residence in 2010, and we’ve been a bonded pair ever since. He never had a lot of dog friends (being part Aussie Cattle Dog / Heeler, he was super bossy and would want to herd them) but was a great people dog. I was a little concerned how he’d transition to condo life (having come from a single family house with a fenced in yard) but he did great.

I miss the little f*cker (a pet name, well earned) a lot, although I’m happy to finally have the opportunity to get ahead of the dog hair and shedding.

Simplex Clock Correction Issues

I’m feeling a bit nostalgic this morning. Readers of a certain age will remember the classic Simplex clocks from school days. The clocks throughout the building had a special feature – a receiver and small control system that would permit the clocks to be synchronized or changed throughout the building, using a master control device. Useful to keep all the clocks at the same time, easily adjust for seasonal daylight savings time changes, and to reset the clocks in the event of power loss.

There’s a good overview of these systems here:

The Synchronous Wire system is the most popular system in the United States. Clocks are run using a power circuit that acts as its time base. The clocks receive an hourly correction which synchronizes both the minute and the second hand. Every 12 hours the clock receives a daily correction to keep the clock perfectly synchronized.

When I first started working in the medical imaging field (circa 1989), we’d run into issues with these clocks, a lot. Hospitals and health-care facilities were big users of these (there’s a clock in every patient room, hallway, and procedure room), and one particular piece of equipment (a Phillips “Classic” generator) was particularly susceptible. The generator used a motor-driven, linear variable autotransformer (think Variac or Powerstat) to adjust for line voltage changes – and the signal injected onto the mains by the clock controller (typically around 3500 Hz) would mess with the voltage regulation circuit, and the motor would “hunt” for the duration of the pulse (usually 5 or 10 seconds), The generator would be disabled or locked out while the motor was moving, and this would drive the docs and techs crazy (since it happened hourly).

Simplex Clocks

A waveform sample from a PowerLines trip report circa 2007, using Rx Monitoring Services power analyzer to capture the clock correction pulse.

 

I’ve also come across a few old power quality threads discussing these clock pulses causing standby / hybrid UPS systems (notoriously sensitive to anything that might indicate the start of an actual outage) to switch to inverter improperly.

Back in the day, the old BMI-4800 power analyzer had a “high frequency noise” detector which looked at broad spectrum harmonics or noise, and output a distinctive “picket fence” 24 hour log when these clocks were present (I still miss this diagnostic / reporting feature on modern power analyzers). I’m sure I have an example of this graph kicking around somewhere but can’t put my paws on it at the moment. I suspect any graphs I recall pre-date digital images (when I would create reports with blank boxes, and manually paste in photo-copied disturance graphs) so I’m not finding anything in trip reports or old PowerPoint presentations)

Resolving these issues? Sometimes we’d consult with the facility engineer – oftentimes these were turned up to “10” (maximum) and we could get the amplitude turned down to the point where it worked but did not cause problems. Sometimes we’d get the clocks reprogrammed to only correct 2x a day (noon and midnight) when it would be unlikely to affect the equipment. Some resourceful field techs developed a filter circuit to protect the regulation circuity; although that was sometimes not permitted (FDA requirements for x-ray equipment forbids modification or retro-fitting).

I don’t hear too much about these lately. Clocks are now often digital, controlled wirelessly or via ethernet. Switched-mode power converters have replaced old analog systems.

 

Cause and Effect: Arcing Transients and Equipment Faults

We recently reviewed some power monitor data for a client. Problem statement:

Breaker Q1 in the WCS electronic box trips on a sporadic basis. The breaker is the M4 and M5 fan motor overcurrent protection. We have replaced the breaker multiple times.

First pass, we noticed three very high current swells in the ground current data:

Ground RMS

We also saw 100s of very serious arcing voltage transients, not related to load current changes or other voltage events. The transients showed up on Phase-Neutral and Neutral-Ground, but the NG transients were much lower amplitude (secondary, not the primary issue).

Transient PhN Transient NG

Finally, we captured three current swell events that clearly show equipment faults to ground (notice the elevated ground current) immediately following voltage transients – cause and effect.

Fault1 Fault2 Fault3

Figuring out the cause of the transients is an exercise for the local service engineers or an onsite power quality engineer. But we’ve got a pretty clear linkage here between transients and equipment faults. Most of the time, power quality problems are a lot less concrete and clear.

How to Annoy Your Friendly Neighborhood Electrical Engineer

Problem Statement: A power assist chair (that we purchased for my mother back in 2016 and no longer need, looking to sell used) had an intermittent – the up/down LED lamps would go out, and the chair would not operate, until one jiggled the power cord.

Annoyance #1: Called the manufacturer to get a replacement cord (I had troubleshot it down the cord itself). Turns out the cord is hard-wired into the motor, so the solution is to replace the motor. (bad answer, the motor itself is fine). Why is a simple power cord that will invariably get a bit of wear and tear not be easily replaceable?

Annoyance #2: Yes, it’s under warranty, but it requires both a service call (to replace the motor) and shipping charges. Um, no.

Annoyance #3: Additionally troubleshot the cable and discovered the intermittent was in a little inline molded box containing an LED, the sole purpose of which (as far as I can see) is as a redundant and unnecessary troubleshooting tool – since both the chair control pendant and the transformer also have LEDs to indicate power on.

Solution: Cut the molded box LED out, strip and splice the cable (nicely soldered and insulated), it’s a wee bit kludgy with electrical tape, but perfectly safe and solid). Took about 15 minutes including time to heat up the soldering iron and then test the crap out of it afterwards.

Ultracomfort America – not impressed.

Audio Project: Buying Local vs. Amazon

I’m in the middle of a small audio project – converting 11 digital micro-cassettes (the sort used in old answering machines and personal recorders) to digital MP3 files for a client’s book / memoir project. It’s potentially slogging work (each cassette has up to 90 minutes of content, that’s up to 16.5 hours of recording) but I’ve got it set up to run in the background, while I do other things. I’m sort of embarrassed to admit that I started doing this on my computer (using Audacity sound recorder / editor) before realizing I have this perfectly good TASCAM DR-07 digital recorder (that I use for live recording) that is actually designed for this sort of thing. Using the computer would have been exceptionally onerous; the digital recorder makes it almost trivial. The only potential problem is battery life on the micro-cassette – I picked up a DC power supply but it introduced severe hum into the signal so I’m back to AAA batteries (I have a stack of those, but I imagine that the batteries will die mid-recording a few times during the process.

I’m probably over-killing the set-up here – running the 2.5mm mono output of the recorder through adapters (2.3mm -> 3.5mm -> 1/4″), then through a DI to get XLR out, and into a Mackie mixer to tweak the sound a bit (cutting out some of the highs and lows, optimizing the level) and into the recorder. I’ve got a set of headphones to listen in now and then. Next time I’m out I’m going to pick up my spare monitor speakers at my storage locker so I can have it going low in the background and lose the headphones.

Getting the adapters was 1/2 the battle here – the 2.5mm mono out is super non-standard. I went to the local Cables & Connectors store which could only supply a 2.5mm stereo, which kind of sort of worked but not really (was a little flaky) – I ended up (as always) finding exactly what I needed at Amazon (2.5mm mono to 3.5mm stereo) and while I was at it, picked up a couple of 3.5mm stereo to 1/4″ breakout cables which I seem to be wanting every other time I pull out the mixer these days (record out to the TASCAM, line out from phones and tablets). I have a bunch of 3.5mm to RCA breakouts but the 1/4 is a lot more mixer friendly.

I like to support local retail when I can, but I’m invariably looking for something a little weird or left of center and rather than drive around all day for something that almost works, just pull up the exact right item on Amazon, order via Prime, and it shows up 1-2 days later. Probably better for the environment as well (considering my gas).