Category Archives: Troubleshooting

Insider stuff, Recent Activity, Troubleshooting, Updates, WED Blog, Windows 11

Vexing Windows 11 Antimalware Platform Update Issues

July 27, 2021 Ed Tittel Leave a comment

Right now, I have two PC dedicated to Windows 11 testing and learning. Just recently, I discovered some vexing Windows 11 Antimalware platform update issues. The short version is: one of my PCs is up-to-date. It’s no longer subject to Automatic Sample Submission reset to off following each restart. Alas, the other remains stubbornly stuck on an earlier Antimalware platform release. None of the update options available work, so I can’t get no relief. Let me explain…

Fighting Vexing Windows 11 Antimalware Platform Update Issues

First, let me be clear. This is a known and documented Windows 11 issue. It’s been around since the initial release hit. Indeed, a fix exists: when the Antimalware Platform version gets to 4.18.2107.4 or higher, the problem disappears. For the record that problem is depicted in this story’s lead-in graphic. After every reboot, the Automatic Sample Submission feature for virus uploads in Defender is turned off. The feature is easy to turn back on, until the next reboot. OCD OS maintainer that I am, the workaround isn’t enough for me. I want it fixed, for good, now.

Here’s the vexing part. WU hasn’t yet deigned to update the antimalware engine behind the scenes. Ditto for the Protection updates option in Windows Security. There’s a registry hack documented on a related ElevenForum thread. There’s even a manual Defender update download that’s supposed to take the Antimalware engine release to 1.2.2107.02. It comes in a file named defender-update-kit-x64.zip. Alas, inspection of said update file shows the Antimalware engine to be 4.18.2015.5. It’s too old to fix the issue, in other words. Thus, no relief just yet, shy of a permanent registry hack.

The Perils of Perfectionism

Yes, I could hack the registry to turn this off. But I’d have to unhack it again when the fix finally shows up on the X380 Yoga that’s affected. I’m going to have to wait for WU to get around to providing me the latest antimalware engine on its own, or find a newer manual update. Alas, that’s the way things go sometimes, here in Windows-World. Oddly, I find myself hoping for a new Windows 11 build, in hopes the latest antimalware engine will be part of its contents. Stay tuned: I’ll let you know how all this shakes out.

Note Added August 4: Update Came!

Thanks to long-time and active TenForums and ElevenForum user @Cliff S, I learned this morning that Antimalware Client Version 4.18.2107.4 arrived via WU. Checking my own previously stuck test machine, I saw it too, had gotten this update. And now, my PC no longer reverts to Automatic Sample Submission=Off after each reboot. Fixed!

I’ve also determined this version is available through the Microsoft Update Catalog. Search for KB4052623, and grab the correct version, if WU doesn’t come through for you.

Insider stuff, Recent Activity, Troubleshooting, Updates, WED Blog, Windows 10

Experience Pack 120.2212.3920.0 Follies

June 24, 2021 Ed Tittel Leave a comment

As it turns out, I should’ve read the Microsoft announcement more carefully. The Windows Insider blog post that announced a new Experience Pack warned me that things would be different for Beta Channel and Release Preview PCs. It said: “For Windows Insiders in the Release Preview Channel, this will be an optional update for you.” I just didn’t pay sufficient attention. And that, dear readers, led me to some unnecessary but still effective Experience Pack 120.2212.3920.0 follies yesterday.

What Kind of Experience Pack 120.2212.3920.0 Follies?

The kind where I decided that because WU didn’t offer my Release Preview PC an obvious and immediate download, I would get it by other means. So, I turned to TenForums.com, where sure enough. I found a thread with a link to a reliable online source. Because this was a .CAB file, I then ran DISM /add-package … to get it installed. It worked!

Then I found out that the Release Preview mechanism differed from the Beta Channel one. Beta Channel (Surface Pro 3) got a direct offer from WU. Release Preview had a new item show up as an “Optional Update” — just as the afore-linked blog post said.

Sigh. One of these days, I’ll slow down and pay more attention. I swear. As Jerry Pournelle used to say in his Byte column from Chaos Manor “Real soon now.” Fortunately, there’s usually more than one path between Points A and B here in Windows-World. Yesterday, mine took me off the beaten track, and had me do manually what WU would have done for me automatically. Sigh again.

I did get here eventually, but not via the most direct route.

One More Thing…

I used DISM to install the KB5004393 update on the Release Preview PC (Lenovo ThinkPad X380). Thus it doesn’t show up in WU Update History (unlike the screencap at the head of this story, which came from the Surface Pro 3). Indeed, I had to go into Programs and Features and use “View installed updates” to find it instead. When you do things manually, reporting changes, too. A word of warning, by way of factual observation.

Networking, Recent Activity, Troubleshooting, WED Blog, Windows 10

Jabbering Transceiver Error Rears Its Ugly Head

June 22, 2021 Ed Tittel Leave a comment

My first real networking job was as a Networking Consultant for Excelan in 1988. That company was purchased in 1989 by Novell, where I stayed quite happily until 1994. My initial training for the position included learning a hardware-based protocol analyzer (the LANalyzer, in fact). One of the things we learned in class was a coax-based 802.1 10 Mbps transceiver could crash an entire physical LAN. This device had a classy alias: “vampire tap.” It was scre-clamped onto a thickwire coax cable to add one or more network ports. Sometimes, its built-in circuitry would go bananas and overrun the network with bogus traffic. This problem, known as a jabbering transceiver error rears its ugly head recently. It happened on one of the Chez Tittel GbE switch domains.

When Jabbering Transceiver Error Rears Its Ugly Head, Divide and Conquer

Here’s a quote from the 2000 classic by Charles Spurgeon: Ethernet: The Definitive Guide

The quote comes courtesy of Google books, pg. 107.
(I still have a hardcopy on my bookshelf).

I’m pretty sure that NICs don’t have transceivers any more, so they aren’t really subject to such failures. But similar behavior — specifically, failure of a switch domain — is well-known to occur when hardware problems bedevil a LAN segment. For a while there, I was chasing random network failures in my office. They would kick all the machines off the switch, but would gradually let everybody back on.

It wasn’t until I quit using the built-in GbE port on my retiring X220 Tablet PC that the problems stopped. I was able to confirm the issue by plugging the RJ-45 cable back into that until and watching the circus start back up. If I switched to a USB dongle instead, the GbE domain attached to either or both switches in my office worked fine. One is a standalone NetGear 8-port GbE switch, the other an 8-port switch integrated into my Asus 802.11AX WAP/router.

Historical Note

Divide and conquer was the recommended troubleshooting method to identify a jabbering transceiver. One would subdivide the cable segment by interrupting it at a repeater, and terminating each sub-segment. Whichever segment stayed broken had the failing device. Repeat until the device can be identified, then replace it. I did this for TRW in Austin in 1988 on an actual service call there…

It wasn’t really until I started the trip down memory lane to my first-ever Ethernet networking class in 1987, and my trip to TRW, that I understood what was happening. The built-in GbE interface was failing, and acting like a jabbering transceiver. I can’t exactly say “everything old is new again.” But I can say, an old lesson learned came in handy. And indeed, that is the way things sometimes go, shooting trouble here in Windows-World!

Insider stuff, Recent Activity, Troubleshooting, Updates, WED Blog, Windows 10

Goodbye Lenovo X220 Tablet PC

June 8, 2021 Ed Tittel Leave a comment

I’ve just learned something potentially useful. As a Windows PC ages, it tends to lose vendor support somewhere along the way. And with that comes missing or incompatible drivers and firmware updates. I’ve hit that point now with my Lenovo X220 Tablet, which was built and purchased in 2012. It was my first-ever touchscreen PC bought to learn touch interaction in Windows 8. But because of increasing decrepitude, I must now say goodbye Lenovo X220 Tablet PC.

Why Say Goodbye Lenovo X220 Tablet PC?

Why? Because it takes longer for me to get the device update ready than it does to apply pending updates. As it’s been a Dev Channel test machine, that’s a lotta updates. Because this phenom includes Defender updates, it’s become a daily thing. Sigh.

I’ve developed a “workaround ritual” to keep the machine updated. First, I try WU by itself. Sometimes, it works. When only Defender updates fail, I next go to the updates button in Windows Security/Virus & Threat protection. If that doesn’t work, I manually download the latest update file and install it “by hand.”

If other updates are involved, I try WUMT. It often succeeds when WU hangs during either download or install phases. Sometimes, I have to reset the entire update environment using Shawn Brink’s Reset_Reregister_Windows_Update_Components.bat file. It’s nearly infallible.

Another problem that’s cropped up is the outright failure of the Intel Management Engine on that PC. I’m not especially worried about that, per se, but this does mean that I must remember to manually strike a key each time the system reboots (and it does so 3 or more times each time any upgrade is installed, which happens weekly on a Dev Channel test machine). Otherwise the system just waits for input before it can proceed further.

When It’s Time, It’s Time…

Long story short, it’s become too time-consuming to work around the X220 Tablet’s limitations and gotchas. I still love this machine, but as a freelancer I always have to keep one eye on the clock and manage my time carefully. This laptop is now more trouble than it’s worth, so I’ll be passing it onto the folks at ReGlue for a wipe and a LInux install. Some schoolkid will still get good use out of its 4-core/8 thread i7 2640M CPU, dual (small) SSDs, and 16 GB RAM.

Recent Activity, Security, Troubleshooting, WED Blog, Windows 10

Pondering IME Recovery State Issues

June 7, 2021 Ed Tittel Leave a comment

OK, then. First let me explain that IME is short for Intel Management Engine. This firmware component is present on all modern PCs with Intel CPUs since 2008. It operates while the OS is active, and IME also runs during boot-up. In fact, IME is accessible even when a PC is shut down or sleeping, as long as power is available. I’m pondering IME recovery state issues for one reason. My 2012-vintage Lenovo X220 Tablet hangs at every restart to report that “ME is in a recovery state.” I must enter a keystroke before boot-up continues.

I’m learning that IME has deep access on any Windows PC where it resides. For more details, check out the Wikipedia article Intel Management Engine.

Why I’m Pondering IME Recovery State Issues

Fixing this issue on my old Lenovo touchscreen PC is proving nearly impossible. Check out this Win-RAID forum thread on ME Cleaner (a management engine cleanup tool). Hopefully, you’ll get a sense of what contortions removing IME entail. Long story short: some real BIOS hacking, with no guarantee of success, is required to disable (or remove) IME at the BIOS level. Sheesh!

The lead-in graphic for this story comes from Intel’s Converged Security and Management Engine Version Detection Tool (CSMEVDT). For the X220 Tablet, it shows that the system is no longer supported (no surprise there, considering its age). No new releases planned, either…

Increasing Horror Results When Pondering IME

In fact, the more I learn about the Intel Management Engine, the more disturbed I become. The Wikipedia article (cited above) does a good job of hitting the high points. What I learned from direct experience on my X220 Tablet is also scary. It goes so far as to speculate that state-level threat actors have been actively seeking out IME exploits for over a decade.

But alas, even after disabling IME in BIOS, the Recovery State error continues. At least the related driver error for “Serial Over LAN” (SOL) access no longer appears in Device Manager.

For the moment, I’m against making BIOS hacks. I’m pretty sure that the absence the SOL driver means IME can no longer access the network. But gosh, this is a scary set of security vulnerabilities to contemplate. Indeed, the rest of my Intel-based systems have IME “working properly.” That’s where my real concerns begin. I’ll have to make sure to patch them all, pronto!

Recent Activity, Tips, Tricks and Tweaks, Troubleshooting, WED Blog, Windows 10

Identifying Windows 10 Mystery Startup Items

June 3, 2021 Ed Tittel Leave a comment

Here’s something new and helpful about working with Task Manager. Take a look at this story’s lead-in graphic. It shows you can right-click any column header in Task Manager’s Startup tab, to see a pick list of columns (checked items). Add the “Startup type” and “Command line” items, and learn more about the startup entries they describe. In fact, they helped me with identifying Windows 10 mystery startup items on my production PC.

When a generic “Program” entry showed up in Startup items, adding fields let me see where it was coming from.
[Click item for full-sized view: see top table entry.]

How-to: Identifying Windows 10 Mystery Startup Items

The “Startup type” tells you where the directive comes from. For “Program” it came from the Registry. Better still, Command line data tells you what Startup executes as Windows 10 gets up and running. The particular instruction is malformed and can’t work:

"C:\Program" Files\Teams Installer\Teams.exe --checkinstall --source=default

The closing double quote is misplaced (it should be at the end of the line). Also the directory path referenced in the command does not actually exist on the PC in which this Registry entry resides.

What did I do about this spurious startup item? I cheerfully disabled it. Indeed, that means there’s an orphaned key-value pair in my registry. I can live with that. I do intend to report it via the Feedback Hub, because it definitely includes a syntax error (the misplaced closing double quote). Otherwise, though, it’s no big deal and I’m satisfied to disable it.

[NOTE} Here’s a shout-out to WinHelpOnline, whose story What is “Program” in Task Manager Startup Tab helped me understand my mystery item. It’s worth reading in its entirety for those who want to learn further details about what’s going on, and how to remove related orphaned registry items.

Insider stuff, Troubleshooting, Updates, WED Blog, Windows 10, Windows Update

Old PC Shows Interesting Update Behaviors

June 2, 2021 Ed Tittel Leave a comment

I’m still running my 2012 vintage Lenovo X220 Tablet. It’s so old, it’s got an Ivy Bridge CPU (i7-2640M). I’ve been getting signs for the past year or so that this PC is nearing obsolescence. For one thing, the Intel Management Engine always comes up in a “recovery state” which I’ve learned means the related firmware is no longer working. In the past month or so, this old PC shows interesting update behaviors. That means it often hangs during update downloads at 0% complete, especially for Windows Defender Security Intelligence updates. Take a look at the lead-in graphic to see what I mean (reproduced below so you can click on it to see all the details).

Click on image for full-sized view.

What Old PC Shows Interesting Update Behaviors Truly Means

Simply put, Windows Update isn’t working reliably on this PC any more. This has persisted across the last half-dozen or so Dev Channel upgrades. The only way to break the logjam seems to be to bring an old tool into the mix — namely, the Windows Update Management Tool (aka WUMT).

If you look at the lines from that application dated June 2 in the lead-in graphic, you’ll get an idea of what’s going on. Notice, the third line from the top shows Defender update failed from MoUpdateOrchestrator. That’s the native service inside WU that coordinates automatic updates. Next, WUMT itself fails (because I actually launched it AFTER firing off a manual update scan in Windows Security’s Virus & Threat protection). That shows up as Windows Defender under “Applications ID” in the top item, and is the one that succeeded.

What Makes This Update Behavior Interesting?

As you can see in the update history, none of the update agents (apps) always succeeds. Sometimes, MoUpdateOrchestrator (WU itself) works. Ditto for Windows Defender and WUMT. I keep using WUMT, though, because it seems to break the 0% download logjam pretty reliably (even if it doesn’t always end doing the download itself, as the lead-in graphic shows).

I am getting a strong sense that the X220 Tablet is nearing the end of its useful life. That’s because I’m deliberately using it to push the envelope to see how well aging hardware copes with Dev Channel Insider Preview builds. When it becomes more work to troubleshoot and get upgraded, I’ll give this machine to my friends at ReGlue and promote one of my two 2018 vintage Lenovo X380 Yoga PCs into that role. If the X220 Tablet is any indication, they should be good for at least another 6 years or so!

Networking, Recent Activity, Tips, Tricks and Tweaks, Troubleshooting, WED Blog, Windows 10

Power Options VM Surprise

May 28, 2021 Ed Tittel Leave a comment

It’s been a painful last few days here in Windows World. I’ve been working on a loaner, locked-down machine in connection with a code analysis project. Because that code is protected and valuable intellectual property (IP), I’m able to access its GitHub repository only through a VM running on a hardened and isolated system. Essentially, I have to access the VM through a browser tab set up inside a VPN-accessible secure store. It hasn’t been going too well, either: each time I tried to use the VM and left the machine alone for a while, it would drop its connection. And then, to make things worse, I couldn’t get back in without asking an IT admin to reset the server side of the remote access environment. That’s where an unwanted and unexepected Power Options VM surprise came into play.

What Is a Power Options VM Surprise?

If you look at the lead-in graphic, you’ll see that one change I make on my Windows PCs post-upgrade or install is to change the sleep interval to “Never.” The default is 30 minutes. Accessing the VM used a commercial VPN into a host server. Then, a remote access client (first RDP, then VNC) connected to the VM itself. For a long time, the firm’s IT guy kept fiddling with RDP settings and such. Eventually he switched to VNC for remote access, thinking it might be an RDP protocol issue at work (or not).

But the disconnect issues kept popping up, where the VM connection would drop when the machine was idle for 30 minutes or more. This finally caused him to investigate the Power Options, where it was immediately obvious the default “sleep after 30 minutes” was the culprit. Resetting the value to my usual preference — that is “Never” — has since fixed things, hopefully for good.

Troubleshooting 101: Don’t Overlook the Obvious

As an outsider with only a regular user account, it wasn’t up to me to mess with default settings on the locked-down machine furnished to me for this project. Ditto for default settings for the VM I was accessing to get into the target code base. But gosh: I have to believe we were looking for complex solutions to a seemingly complex problem. Instead, we should have been looking for simple solutions for a straightforward default settings check.

The moral of this story is not lost on me. I hope it will likewise inspire you to make a checklist when working with VMs, and to put “check default settings” (especially in Power Options) right near the head of that list. Sleep may “knit up the raveled sleeve of care,” as the Immortal Bard put it. But sleep causes all kinds of interesting problems for Windows PC — and now I know, for Windows VMs, too. Funny thing, I’ve learned to make this tweak because I use RDP extensively here at Chez Tittel to get from my production desktop to the 10-plus other PCs usually running around here. I shoulda known…

Recent Activity, Tips, Tricks and Tweaks, Troubleshooting, Updates, WED Blog, Windows 10

Blinking Monitor Gets Easy Fix

May 26, 2021 Ed Tittel Leave a comment

When it comes to Windows, it’s always something. When I logged in this morning, it was my number two (right-hand) monitor, blinking on and off at about 3 second intervals. From long experience, I know the most likely cause for such misbehavior is the graphics driver. Thus, I immediately fire up the GeForce Experience app, see a new driver is available, download and install same. And that, dear Readers, is how my blinking monitor gets easy fix. If only all of my problems were so easily solved!

Driver Update Means Blinking Monitor
Gets Easy Fix

Graphics drivers are notoriously finicky beasts. They can cause all kinds of interesting problems, especially when new drivers cause hijinks on older graphics cards (or circuitry). My production desktop incorporates a GeForce GTX 1070, which is now about 5 years old. Because of the scarcity of newer generation (2xxx and 3xxx) GPUs right now — coin miners are snatching them up in droves — this model is still in extremely wide use. Hence, I’m inclined to trust new drivers. That’s because Nvidia would aggravate a sizable population if they let a substandard GTX 1070 driver out the door.

Luckily for me, my inclinations proved justified. After installing v466.47, I see no further blinking from the right-hand monitor (#2 in the lead-in graphic). It’s nice when the most obvious fix turns out to be the only one that’s required. Again, I know from experience that troubleshooting issues further would get more interesting and probably end up costing money.

My next move would have been to swap the DisplayPort cables that tie monitors 1 and 2 to the GeForce card. If the blinking had switched positions, that would indicate a cable replacement. If not, card troubleshooting would begin in earnest. And with GPUs so expensive and hard to find right now, that could have been a real problem.

Sometimes, here in Windows-World, you get away with an occasionally easy fix for your problems. Today, I’m celebrating my simple and painless escape!

Recent Activity, Troubleshooting, Updates, WED Blog, Windows 10, Windows Update

Is Forcing Win10 Upgrades Good?

May 21, 2021 Ed Tittel Leave a comment

After my amazing experience in forcibly upgrading the Lenovo X12 hybrid tablet yesterday I’m pondering upgrade strategies. Indeed, 2004 and 20H2 Windows 10 PCs are in line for the 21H1 upgrade. But Microsoft’s criteria for offering that upgrade — and thus also, its timing — are unclear. Hence my question: “Is forcing Win10 upgrades good?” As is the case with most good questions, the answer starts with a predictable phrase: “That depends…”

Answering “Is Forcing Win10 Upgrades Good?”

I got to 21H1 on the X12 by downloading a self-installing upgrade file (.MSU) from a link at TenForums.com. Here’s what that info looks like on that page (links are not live, and you’ll soon understand why):

These catalog downloads no longer show up when you search the catalog, but they’re still live.
[Click image for full-sized view.]

Those links do work (I’ve checked) and they come from download.microsoft.com, which is indeed the Update Catalog’s home. But a search on KB5000736 comes up dry. So MS is not offering this enablement package directly from the catalog anymore. That does suggest that the answer to this article’s main question is “If it works, then it’s good; if not, then it’s not.”

Expect the Best, But Prepare for the Worst

Because MS isn’t providing the enablement package directly as a catalog download, that means MS wants you to wait for Windows Update to make the offer. If you choose (as I did) to skip the wait and grab the enablement package from an alternate source (ditto), you should follow the sub-title’s advice. That is, I’d recommend making an image backup before applying the MSU file. Then, if the upgrade fails, you can boot to repair/recovery media and replace the current, suspect image with a current, known good working replacement.

The ISO files for 21H1 are also available. The great appeal of the enablement package is that it’s blazing fast. If you do the ISO route, you’ll run setup.exe from its root folder and it will be a typical upgrade. The experience takes at least 15 minutes to complete, and leaves the Windows.old folder hierarchy around so you can roll back to 20H2 or 2004 as you might like. In that way, it may be “safer” than forcing the enablement package onto a PC. That’s because recovery from failure will be automatic, and you can even elect to roll back up to 10 days afterward if you decide you don’t like where 21H2 takes your PC.

Same Question, Different Answer

Another way to ponder the question “Is Forcing Win10 Upgrades Good?” is to try it, and see what happens. If it works, then yes. If it doesn’t, not only is the answer no, but your subsequent experience will depend on whether or not your pre-planning includes a recovery path. If it doesn’t the answer is “No, and it’s a PITA;” if it does, the answer is “No, but it didn’t take too long or hurt too much.”

And that, dear readers, is the way things sometimes go here in Windows World. it also explains why I still haven’t forced the enablement package onto my production PC just yet. I’m still thinking…