Category Archives: Troubleshooting

Goodbye Lenovo X220 Tablet PC

I’ve just learned something potentially useful. As a Windows PC ages, it tends to lose vendor support somewhere along the way. And with that comes missing or incompatible drivers and firmware updates. I’ve hit that point now with my Lenovo X220 Tablet, which was built and purchased in 2012. It was my first-ever touchscreen PC bought to learn touch interaction in Windows 8. But because of increasing decrepitude, I must now say goodbye Lenovo X220 Tablet PC.

Why Say Goodbye Lenovo X220 Tablet PC?

Why? Because it takes longer for me to get the device update ready than it does to apply pending updates. As it’s been a Dev Channel test machine, that’s a lotta updates. Because this phenom includes Defender updates, it’s become a daily thing. Sigh.

I’ve developed a “workaround ritual” to keep the machine updated. First, I try WU by itself. Sometimes, it works. When only Defender updates fail, I next go to the updates button in Windows Security/Virus & Threat protection. If that doesn’t work, I manually download the latest update file and install it “by hand.”

If other updates are involved, I try WUMT. It often succeeds when WU hangs during either download or install phases. Sometimes, I have to reset the entire update environment using Shawn Brink’s Reset_Reregister_Windows_Update_Components.bat file. It’s nearly infallible.

Another problem that’s cropped up is the outright failure of the Intel Management Engine on that PC. I’m not especially worried about that, per se, but this does mean that I must remember to manually strike a key each time the system reboots (and it does so 3 or more times each time any upgrade is installed, which happens weekly on a Dev Channel test machine). Otherwise the system just waits for input before it can proceed further.

When It’s Time, It’s Time…

Long story short, it’s become too time-consuming to work around the X220 Tablet’s limitations and gotchas. I still love this machine, but as a freelancer I always have to keep one eye on the clock and manage my time carefully. This laptop is now more trouble than it’s worth, so I’ll be passing it onto the folks at ReGlue for a wipe and a LInux install. Some schoolkid will still get good use out of its 4-core/8 thread i7 2640M CPU, dual (small) SSDs, and 16 GB RAM.

 

Facebooklinkedin
Facebooklinkedin

Pondering IME Recovery State Issues

OK, then. First let me explain that IME is short for Intel Management Engine. This firmware component is present on all modern PCs with Intel CPUs since 2008. It operates while the OS is active, and IME also runs during boot-up. In fact, IME is accessible even when a PC is shut down or sleeping, as long as power is available. I’m pondering IME recovery state issues for one reason. My 2012-vintage Lenovo X220 Tablet hangs at every restart to report that “ME is in a recovery state.” I must enter a keystroke before boot-up continues.

I’m learning that IME has deep access on any Windows PC where it resides. For more details, check out the Wikipedia article Intel Management Engine.

Why I’m Pondering IME Recovery State Issues

Fixing this issue on my old Lenovo touchscreen PC is proving nearly impossible. Check out this Win-RAID forum thread on ME Cleaner (a management engine cleanup tool). Hopefully, you’ll get a sense of what contortions removing IME entail. Long story short: some real BIOS hacking, with no guarantee of success, is required to disable (or remove) IME at the BIOS level. Sheesh!

The lead-in graphic for this story comes from Intel’s Converged Security and Management Engine Version Detection Tool (CSMEVDT). For the X220 Tablet, it shows that the system is no longer supported (no surprise there, considering its age). No new releases planned, either…

Increasing Horror Results When Pondering IME

In fact, the more I learn about the Intel Management Engine, the more disturbed I become. The Wikipedia article (cited above) does a good job of hitting the high points. What I learned from direct experience on my X220 Tablet is also scary. It goes so far as to speculate that state-level threat actors have been actively seeking out IME exploits for over a decade.

But alas, even after disabling IME in BIOS, the Recovery State error continues. At least the related driver error for “Serial Over LAN” (SOL) access no longer appears in Device Manager.

For the moment, I’m against making BIOS hacks. I’m pretty sure that the absence the SOL driver means IME can no longer access the network. But gosh, this is a scary set of security vulnerabilities to contemplate. Indeed, the rest of my Intel-based systems have IME “working properly.” That’s where my real concerns begin. I’ll have to make sure to patch them all, pronto!

Facebooklinkedin
Facebooklinkedin

Identifying Windows 10 Mystery Startup Items

Here’s something new and helpful about working with Task Manager. Take a look at this story’s lead-in graphic. It shows you can right-click any column header in Task Manager’s Startup tab, to see a pick list of columns (checked items). Add the “Startup type” and “Command line” items, and learn more about the startup entries they describe. In fact, they helped me with identifying Windows 10 mystery startup items on my production PC.

Identifying Windows 10 Mystery Startup Items.program

When a generic “Program” entry showed up in Startup items, adding fields let me see where it was coming from.
[Click item for full-sized view: see top table entry.]

How-to: Identifying Windows 10 Mystery Startup Items

The “Startup type” tells you where the directive comes from. For “Program” it came from the Registry. Better still, Command line data tells you what Startup executes as Windows 10 gets up and running. The particular instruction is malformed and can’t work:

"C:\Program" Files\Teams Installer\Teams.exe --checkinstall --source=default

The closing double quote is misplaced (it should be at the end of the line). Also the directory path referenced in the command does not actually exist on the PC in which this Registry entry resides.

What did I do about this spurious startup item? I cheerfully disabled it. Indeed, that means there’s an orphaned key-value pair in my registry. I can live with that. I do intend to report it via the Feedback Hub, because it definitely includes a syntax error (the misplaced closing double quote). Otherwise, though, it’s no big deal and I’m satisfied to disable it.

[NOTE} Here’s a shout-out to WinHelpOnline, whose story What is “Program” in Task Manager Startup Tab helped me understand my mystery item. It’s worth reading in its entirety for those who want to learn further details about what’s going on, and how to remove related orphaned registry items.

Facebooklinkedin
Facebooklinkedin

Old PC Shows Interesting Update Behaviors

I’m still running my 2012 vintage Lenovo X220 Tablet. It’s so old, it’s got an Ivy Bridge CPU (i7-2640M). I’ve been getting signs for the past year or so that this PC is nearing obsolescence. For one thing, the Intel Management Engine always comes up in a “recovery state” which I’ve learned means the related firmware is no longer working. In the past month or so, this old PC shows interesting update behaviors. That means it often hangs during update downloads at 0% complete, especially for Windows Defender Security Intelligence updates. Take a look at the lead-in graphic to see what I mean (reproduced below so you can click on it to see all the details).

Old PC Shows Interesting Update Behaviors
Old PC Shows Interesting Update Behaviors

Click on image for full-sized view.

What Old PC Shows Interesting Update Behaviors Truly Means

Simply put, Windows Update isn’t working reliably on this PC any more. This has persisted across the last half-dozen or so Dev Channel upgrades. The only way to break the logjam seems to be to bring an old tool into the mix — namely, the Windows Update Management Tool (aka WUMT).

If you look at the lines from that application dated June 2 in the lead-in graphic, you’ll get an idea of what’s going on. Notice, the third line from the top shows Defender update failed from MoUpdateOrchestrator. That’s the native service inside WU that coordinates automatic updates. Next, WUMT itself fails (because I actually launched it AFTER firing off a manual update scan in Windows Security’s Virus & Threat protection). That shows up as Windows Defender under “Applications ID” in the top item, and is the one that succeeded.

What Makes This Update Behavior Interesting?

As you can see in the update history, none of the update agents (apps) always succeeds. Sometimes, MoUpdateOrchestrator (WU itself) works. Ditto for Windows Defender and WUMT. I keep using WUMT, though, because it seems to break the 0% download logjam pretty reliably (even if it doesn’t always end doing the download itself, as the lead-in graphic shows).

I am getting a strong sense that the X220 Tablet is nearing the end of its useful life. That’s because I’m deliberately using it to push the envelope to see how well aging hardware copes with Dev Channel Insider Preview builds. When it becomes more work to troubleshoot and get upgraded, I’ll give this machine to my friends at ReGlue and promote one of my two 2018 vintage Lenovo X380 Yoga PCs into that role. If the X220 Tablet is any indication, they should be good for at least another 6 years or so!

Facebooklinkedin
Facebooklinkedin

Power Options VM Surprise

It’s been a painful last few days here in Windows World. I’ve been working on a loaner, locked-down machine in connection with a code analysis project. Because that code is protected and valuable intellectual property (IP), I’m able to access its GitHub repository only through a VM running on a hardened and isolated system. Essentially, I have to access the VM through a browser tab set up inside a VPN-accessible secure store. It hasn’t been going too well, either: each time I tried to use the VM and left the machine alone for a while, it would drop its connection. And then, to make things worse, I couldn’t get back in without asking an IT admin to reset the server side of the remote access environment. That’s where  an unwanted and unexepected Power Options VM surprise came into play.

What Is a Power Options VM Surprise?

If you look at the lead-in graphic, you’ll see that one change I make on my Windows PCs post-upgrade or install is to change the sleep interval to “Never.” The default is 30 minutes. Accessing the VM used a commercial VPN into a host server. Then, a remote access client (first RDP, then VNC) connected to the VM itself. For a long time, the firm’s IT guy kept fiddling with RDP settings and such. Eventually he switched to VNC for remote access, thinking it might be an RDP protocol issue at work (or not).

But the disconnect issues kept popping up, where the VM connection would drop when the machine was idle for 30 minutes or more. This finally caused him to investigate the Power Options, where it was immediately obvious the default “sleep after 30 minutes” was the culprit. Resetting the value to my usual preference — that is “Never” — has since fixed things, hopefully for good.

Troubleshooting 101: Don’t Overlook the Obvious

As an outsider with only a regular user account, it wasn’t up to me to mess with default settings on the locked-down machine furnished to me for this project. Ditto for default settings for the VM I was accessing to get into the target code base. But gosh: I have to believe we were looking for complex solutions to a seemingly complex problem. Instead, we should have been looking for simple solutions for a straightforward default settings check.

The moral of this story is not lost on me. I hope it will likewise inspire you to make a checklist when working with VMs, and to put “check default settings” (especially in Power Options) right near the head of that list. Sleep may “knit up the raveled sleeve of care,” as the Immortal Bard put it. But sleep causes all kinds of interesting problems for Windows PC — and now I know, for Windows VMs, too. Funny thing, I’ve learned to make this tweak because I use RDP extensively here at Chez Tittel to get from my production desktop to the 10-plus other PCs usually running around here. I shoulda known…

Facebooklinkedin
Facebooklinkedin

Blinking Monitor Gets Easy Fix

When it comes to Windows, it’s always something. When I logged in this morning, it was my number two (right-hand) monitor, blinking on and off at about 3 second intervals. From long experience, I know the most likely cause for such misbehavior is the graphics driver. Thus, I immediately fire up the GeForce Experience app, see a new driver is available, download and install same. And that, dear Readers, is how my blinking monitor gets easy fix. If only all of my problems were so easily solved!

Driver Update Means Blinking Monitor
Gets Easy Fix

Graphics drivers are notoriously finicky beasts. They can cause all kinds of interesting problems, especially when new drivers cause hijinks on older graphics cards (or circuitry). My production desktop incorporates a GeForce GTX 1070, which is now about 5 years old. Because of the scarcity of newer generation (2xxx and 3xxx) GPUs right now — coin miners are snatching them up in droves — this model is still in extremely wide use. Hence, I’m inclined to trust new drivers. That’s because Nvidia would aggravate a sizable population if they let a substandard GTX 1070 driver out the door.

Luckily for me, my inclinations proved justified. After installing v466.47,  I see no further blinking from the right-hand monitor (#2 in the lead-in graphic). It’s nice when the most obvious fix turns out to be the only one that’s required. Again, I know from experience that troubleshooting issues further would get more interesting and probably end up costing money.

My next move would have been to swap the DisplayPort cables that tie monitors 1 and 2 to the GeForce card. If the blinking had switched positions, that would indicate a cable replacement. If not, card troubleshooting would begin in earnest. And with GPUs so expensive and hard to find right now, that could have been a real problem.

Sometimes, here in Windows-World, you get away with an occasionally easy fix for your problems. Today, I’m celebrating my simple and painless escape!

Facebooklinkedin
Facebooklinkedin

Is Forcing Win10 Upgrades Good?

After my amazing experience in forcibly upgrading the Lenovo X12 hybrid tablet yesterday I’m pondering upgrade strategies. Indeed, 2004 and 20H2 Windows 10 PCs are in line for the 21H1 upgrade. But Microsoft’s criteria for offering that upgrade — and thus also, its timing — are unclear. Hence my question: “Is forcing Win10 upgrades good?” As is the case with most good questions, the answer starts with a predictable phrase: “That depends…”

Answering “Is Forcing Win10 Upgrades Good?”

I got to 21H1 on the X12 by downloading a self-installing upgrade file (.MSU) from a link at TenForums.com. Here’s what that info looks like on that page (links are not live, and you’ll soon understand why):

Is Forcing Win10 Upgrades Good? Catalog Links

These catalog downloads no longer show up when you search the catalog, but they’re still live.
[Click image for full-sized view.]

Those links do work (I’ve checked) and they come from download.microsoft.com, which is indeed the Update Catalog’s home. But a search on KB5000736 comes up dry. So MS is not offering this enablement package directly from the catalog anymore. That does suggest that the answer to this article’s main question is “If it works, then it’s good; if not, then it’s not.”

Expect the Best, But Prepare for the Worst

Because MS isn’t providing the enablement package directly as a catalog download, that means MS wants you to wait for Windows Update to make the offer. If you choose (as I did) to skip the wait and grab the enablement package from an alternate source (ditto), you should follow the sub-title’s advice. That is, I’d recommend making an image backup before applying the MSU file. Then, if the upgrade fails, you can boot to repair/recovery media and replace the current, suspect image with a current, known good working replacement.

The ISO files for 21H1 are also available. The great appeal of the enablement package is that it’s blazing fast. If you do the ISO route, you’ll run setup.exe from its root folder and it will be a typical upgrade. The experience takes at least 15 minutes to complete, and leaves the Windows.old folder hierarchy around so you can roll back to 20H2 or 2004 as you might like. In that way, it may be “safer” than forcing the enablement package onto a PC. That’s because recovery from failure will be automatic, and you can even elect to roll back up to 10 days afterward if you decide you don’t like where 21H2 takes your PC.

Same Question, Different Answer

Another way to ponder the question “Is Forcing Win10 Upgrades Good?” is to try it, and see what happens. If it works, then yes. If it doesn’t, not only is the answer no, but your subsequent experience will depend on whether or not your pre-planning includes a recovery path. If it doesn’t the answer is “No, and it’s a PITA;” if it does, the answer is “No, but it didn’t take too long or hurt too much.”

And that, dear readers, is the way things sometimes go here in Windows World. it also explains why I still haven’t forced the enablement package onto my production PC just yet. I’m still thinking…

Facebooklinkedin
Facebooklinkedin

SetupDiag Illuminates Updates Too

About three months ago I wrote about the Microsoft SetupDiag.exe tool. In that February 17 post, I explained how it provides info about upgrade errors and gotchas. Although the Microsoft Docs article doesn’t really say so, SetupDiag Illuminates Updates too. That is: you can use it to gather information and intelligence about update errors, failures, and so forth. Because those occur more frequently than upgrades, this capability is perhaps even more valuable.

If SetupDiag Illuminates Updates Too, Then What?

A failed Windows Upgrade leaves a copy of SetupDiag.exe behind, in the $Windows.~BT/Sources folder. Windows Update does no such thing. Thus, would-be investigators should bookmark this link, from whence the latest and greatest version may always be downloaded:

Download SetupDiag

Once you have this tool in hand, open an administrative Command Prompt or PowerShell session, then enter its full path specification. I found one in the Windows.old folder hierarchy on a recently-upgraded Dev Channel test PC, and it produced the following (partial) output:

SetupDiag Illuminates Updates Too.output-example

Run a local copy of the program if you’ve got one, though it’s best to download a current version instead.
[Click image for full-sized view.]

Once SetupDiag runs through all of its log searches and processing rules, it will produce a report that provides the error code and error string (aka “bug check code” and “bug check string,” respectively). This is usually enough information to lead affected users to possible solutions. Just today, in fact, I read a story about update failures for the May 11 KB5003173 that used such data to diagnose possible issues with manual Microsoft Edge removals. It seems that leaving old directories behind will stymie the update. See this Windows Latest story for details.

The Consummation You Should Seek

Be it upgrade or update, you’ll eventually want SetupDiag to show you something like this to indicate a successful outcome:

Once you’ve finished troubleshooting, and fixed things, SetupDiag should tell you something like this.
[Click image for full-sized view.]

Cheers!

Facebooklinkedin
Facebooklinkedin

Resenting Mobile-Only Network UIs

In the process of troubleshooting the LAN here at the Tittel household recently, I replaced a Gen 1 Router/Switch/WAP device with its Gen 2 counterpart. Spectrum provides that equipment for something like $7 a month. I don’t resent that charge. But what has me considering a switch to my own equipment is resenting mobile-only network UIs now forced upon me.

What does that mean? It means I can’t log into the gateway’s TCP/IP address in a Web browser any more to access and manage setup and configuration. No! I must now use the MySpectrum app on a cellphone instead. That’s a problem for all kinds of reasons, some good ones, and some that may sound whiny. Apologies in advance.

Why I’m Resenting Mobile-Only Network UIs

Because I MUST run the device UI through a smartphone app, I’m limited to its tiny screen, virtual touch keyboard, and limited silicon. Basically, that means my 100 wpm typing speed on a keyboard falls off  a cliff when I switch to a screen-based layout. This gives my facility and productivity a massive knock, and earns my displeasure.

And alas, I’m no spring chicken anymore either. At 68, I am already in the habit of viewing web pages at 125% magnification to make things easier on my eyeballs. I’ve been known to bump that to 150% or higher when faced with lots of fine print. Forcing me onto a 750×1334 screen goes against my preferences, and hurts my eyes.

And then there are the UI exigencies that small screens dictate. I checked, and I have to work through 7 screens to reserve an IP address within the new app. It used to be a lot faster and easier under the old, Web-based UI. Sigh.

Now that my rant is ended, I’d like to remind Spectrum that good customer service is about providing accessible alternatives. C’mon guys: if a late middle-aged, early geriatric has mild usability issues, what about others with more severe access or vision impediments? Is a cellphone-only approach really workable for everybody?

Exploring Technology Alternatives

I won’t let this slow me down too much. First, I plan to see if I can get MySpectrum to run on my iPad. I do have a Bluetooth keyboard for that device, and can put it to work for configuration jobs. I also see that long-time high-value remote access app TeamViewer lets users run a cellphone app from a PC desktop. That’s not the usual path for remote access between such devices, but it might be just what I need.

Stay tuned. Once my current fit of pique subsides, I may find some kind of workable alternative or usage scenario. If I do, I’ll report back with more info.

Facebooklinkedin
Facebooklinkedin

NirSoft BlueScreenView Worth Learning

Israeli developer Nir Sofer is the person behind the outstanding Windows utility site nirsoft.net. I’ll be describing his blue screen viewing tool in today’s item. And when I explain what makes NirSoft BlueScreenView worth learning, I mean it is something handy to have around for both Windows professionals and enthusiasts.

Why say this? Because, sooner or later, nearly every Windows PC experiences a crash. In older Windows versions, such a screen was invariably blue. That earned it the initialism BSOD, for “Blue Screen of Death.” In Windows 10, such screens sometimes come up in green instead and may be called GSODs for that reason. For a fascinating historical look at BSODs from the past, check out Mark Russinovich’s evil little BlueScreen Screen Saver. It not only simulates BSODs, it also simulates the data acquisition and reboot phases that follow immediately thereafter.  Says Russinovich “…its accuracy will fool even advanced NT developers” (it does not, however, look like a real Windows 10 BSOD or GSOD). Like I said: it’s evil.

Why Is NirSoft BlueScreenView Worth Learning?

Simply put, this nice little tool reads the dump files that Windows collects as it recovers from a serious error. It provides immediate insight into what blew up, and what other OS and application modules were involved.

You can provoke BSOD with an input string to an administrative command prompt, if you like. WARNING! This will immediately crash the PC into which it is entered. Close all apps, and save your work beforehand, to avoid unpleasant surprises.

That command string is:

taskkill /im svchost.exe /f

Svchost.exe is a critical Windows 10 process. It acts as a shell for loading services based around dynamic load library (DLL) files. Because DLLs are often shared, multiple processes will call on a single svchost.exe instance to access its DLL. By running this command you’re killing all svchost instances immediately. This renders Windows unable to run, so it crashes instead.

The flag in the resulting BSOD reads “CRITICAL_PROCESS_DIED.” That brief phrase tells you that, except as a sure-fire way of provoking a BSOD, this is an extremely bad idea. But it’s a useful technique to cause a bluescreen, to show what NirSoft BlueScreenView can do.

NirSoft BlueScreenView Worth Learning.bsd-windows

Dump files in top pane, Dump trace in lower pane. This one shows the CRITICAL_PROCESS_DIED error from the lead-in graphic.
[Click image for full-sized view.]

Working Through BlueScreenView Output

As you examine the image above, you’ll see a dump file that starts with a date string (051021) and ends with the tell-tale file extension “.dmp”. It shows a time stamp, the bug check string, and a bug check code, followed by up to 4 parameters. It also shows which driver caused the crash: in this case, we killed the driver for the operating sytem kernel itself! (That’s noskrnl.exe plus a hex offset, as shown in column 9.)

Generally when I’m using this tool, I look first at Column 9 (caused by driver). That’s because the transitory blue screen window provides most of the preceding data. I usually care most about the bug check string and code because they make dandy lookup strings for guidance online. Column 9 points to the actual cause, and can be extremely informative.

Spend a little time with this tool, and use it to practice reading dump files. Trust me: it’ll come in handy someday. ‘Nuff said

Facebooklinkedin
Facebooklinkedin