The more GUI fails me, the more I find myself back in the CLI.

This is 4 different PowerShell (version 7.5.4) sessions, all running in W11 25H2 Terminal. WHY you ask? Great question, because my Microsoft Excel is currently a gray screen with a blue spinning curser of DEATH! I have almost 1,000,000 rows of data to parse/process/dedup/filter/exclude/prune. This should be fairly simple operation for Excel, but NO!
This is where I find a little AI (ChatGPT GPT-5) and PowerShell come in quite handy. If asked, Python might handle this much faster but I’m not familiar with it and would rely too heavily on AI assistance. At least with PowerShell I can figure out some errors and understand most of the code.
The end game is matching folder permissions from one platform to another at a certain folder hierarchy level. Maybe matching is the wrong word, how about clarifying. I’m taking > 1 million folders and removing a certain level of subfolder and then applying a permission group to a certain folder via a template. The template will run in python on the destination. The issue is the source data is not consistent enough to apply a constant template so I have to manipulate the source data to make it consistent enough. I can’t share more details without revealing too much proprietary information but if you’ve ever moved a massive amount of data (100+ TB) from one platform that spans 20+ years and has NTFS style permissions to another platform that is not NTFS then you might know what I’m referring to.
It is fun to see all the PowerShell windows fly by with the colors and output. I have one screen manipulating the data which are a ton of smaller TXT/CSV files, and another window monitoring the output folder. I also have some checks and balances in place and countdowns and timers and heartbeat so if the process stops I can pick back up where I left off. I also log EVERYTHING. I am also setting up this process to run on different folders for the future.
This is a chain of thought and not well typed. I tried to write the bulk of this on my site but the mobile app is failing me right now.

UPDATE: I had to revert my website from backup 24 hours ago and now I can post. Below is the error I received.

UPDATE: As of the next morning, it had gone from 25 batch files to 125 batch files and still counting…



You should leave a comment. Jason would like you better if you did. Jason will manually approve all comments before they appear.