Feature Request: Parallel Processing (Multithreading)

mkupper

No sure why so long time have passed and no implementation of Parallel Processing or Multithreading. is it difficult to implement or what?. i am having 20 core CPU and just one core Notepad ++ is utilizing and taking forever. please update if this is possible or not possible.

and @ethpsantos said:

Right now I’m doing a Replace in a file with 1 million lines… the CPU used is at max 13% that is just a single core.
It could speed up a lot of tasks that take like 20-30 minutes to do in daily tasks… that is a lot of save.

Nearly all tools, including Notepad++, are single threaded. However, you can start 20 instances of Notepad++, or nearly any tool, and to have them do things in parallel, if that’s your desire. For Notepad++ you can either:

Set up 20 separate portable copies and to start them using the -multiInst command line argument. This provides the best isolation between your copies so that they don’t step on each other.
Set up one portable copy and have 20 separate settings folders. Start your copies using the -settingsDir and -multiInst command line options. This provides nearly the same level of isolation between the copies. Some plugins, such as NppConverter seem to ignore -settingsDir and so this is something you should be alert for when setting up scripts that will be run for each each copy of Notepad++.

NppExec and PythonScript, and probably other plugins, support running a script when Notepad++ is started. That’s how you will get your 20 copies of Notepad++ to do whatever you want as they are started.

As you will have 20 config.xml files you can adjust each of the 20 copies to have a small window on your monitor(s) so that you can view the progress of all 20 copies doing things at once.

Harry Brookes

@gstavi said in Feature Request: Parallel Processing (Multithreading):

A single CPU has enough power to handle your keyboard typing.

“640K ought to be enough for anybody”

Only 3 years ago but someone with similar foresight to half a century ago.

There’s always someone there to explain why something hasn’t been implemented yet, rather than questioning “why not?”

Some of Notepad++ users use it as a reverse engineering tool and when running “Find in files” (great feature) which is extremely slow in comparison to even a 10 year old mobile phone (yes lecture me on architectures or if I’m lucky why a reduced instruction-set computer outperforms x86 at this task that’s 10 generations ahead).

The team at Notepad++ are amazing for their contributions and development of this app but comments like yours only stifle development. 8 people boosted up a comment because they don’t need it themselves… So the people who ask for sensible 21st century advancements get pushed to the side.

Your comment started with “Most of us”… I’ll answer that "All of us will benefit from multi-threaded workloads…

If they can support 32-bit builds which next to none of us use a 32-bit computer (which even then they have multiple cores)… Then asking for multi-threading support on a find and replace really isn’t asking a great deal, is it?

Harry Brookes

@ThosRTanner I’ve just achieved multi-threaded find and replace in a simple Powershell script using 1 thread per directory before moving onto the next job.

I think it’s simpler when people don’t try to find reasons for things being harder than they actually are.

Powershell is one of the most primitive “high-level” languages around and it was much faster.

Of course a pretty GUI with better ease-of-use is much more preferrable :)

Mark Olson

@Harry-Brookes said in Feature Request: Parallel Processing (Multithreading):

I think it’s simpler when people don’t try to find reasons for things being harder than they actually are.

People constantly insist that the single-threaded nature of most core Notepad++ APIs must just be due to sheer laziness on the part of its core devs, and those people have apparently never actually looked at the codebase and tried to understand the issues involved. Refactoring anything in NPP to use multiple threads is (by my estimate) several times more difficult than creating a multithreaded version of the same thing from the ground up.

Notepad++ has a tremendous amount of shared global state, which could only be made compatible with multithreading through extensive use of mutexes, which would then introduce dramatic performance losses in single-threaded use cases (at best) and deadlock (at worst). By the way, have you ever tried to debug a multithreaded program without extensive use of log files? Hahahahaha, good luck.

Find-and-replace in files seems like an embarrassingly parallel operation (or at least one that can be achieved with only one or two mutexes), but the devil is in the details, and as someone who has actually read the source code for Notepad++'s implementation, there are a lot of details there.

EDIT: AFAICT, the simplest way to even begin to implement multithreaded find-in-files in NPP would be to use multiple Scratchtillas (NPP uses _pscratchTilla as a lightweight way to run Scintilla operations like find/replace on a file without doing any graphics rendering), and use some simple task-dividing/scheduling algorithm to decide which Scratchtilla works with which file. The scariest problem that I can think of comes when the the Scratchtilla has to report its progress back to the form that shows progress. From my limited experience (I am admittedly a noob to multithreading with GUIs), there seems to be a lot of risk of deadlock whenever multiple threads contend for a lock on a GUI element. And that’s just the tip of the iceberg.

Harry Brookes

@Mark-Olson Embarrasing to yourself but maybe not for people wanting to reverse engineer 100,000s of files to find matches.

Harry Brookes

@Mark-Olson To use Scractchtilla is a useful insight. Ive used external tools that I’ve automated to get multithreading capabilities.

At least me mentioning here, opened up the topic.

Rewriting the entire NP++ project is no doubt not viable. I won’t accept from anyone here that there isn’t a way to make plugins etc capable of utilising multithreaded ops.

Maybe my reasoning for using NP++ is different to other users in the forum… It’s only one tool in the toolbox atm.

I’ve used simple tools to find n search through 100,000s of files (that always becomes a “Not Responding” NP++ instance… To get the job done.

NP++ is still in my toolkit. I was only suggesting improvemeents.

“It can’t be done” - It doesn’t sit right with me

PeterJones

@Harry-Brookes said:
“It can’t be done” - It doesn’t sit right with me

He didn’t say that. Two years ago, he said “the devil is in the details”, which means it’s a pain to get it right, not that it can’t be done.

I won’t accept from anyone here that there isn’t a way to make plugins etc capable of utilising multithreaded ops.

Again, putting words into people’s mouths vs what they actually said, two years ago. I cannot find any claim above that plugins couldn’t do multithreading.

And, in fact, in that intervening two years, @coises has created the Search++ plugin, which is trying, among other things, to add multiple threads (at least for Find In Files). You might want to try that plugin, and see if the FiF performance is better for you there… That discussion is still active, so if you have specific feedback, that would be a good place for it.

Coises

@harry-brookes , @peterjones

“the devil is in the details”

And there are more details than I knew existed. The details have details.

It’s easy for a naïve multi-threaded implementation to be slower than a single-threaded implementation, as was demonstrated when @guy038 ran a test of my search on a folder on a USB stick.

I’m working now on improving it. It’s incredible how many ways there are to shoot yourself in the foot with multi-threading. I’ll feel validated if I can come up with something that is sometimes better than single-threaded and never worse than single-threaded. It remains to be seen if I can do it.

gstavi

Professionals who search 100,000s of files with Notepad++ are not professionals.
Professionals, when they reach that scale (and long before), use command line tools. Then they open the found files in an editor. Possibly Notepad++.

Haven’t looked at Notepad++ code for many years and have no plans to do so anytime soon.
The main performance bottleneck of search in files was that every file was fully loaded into Notepad++, then its encoding was guessed. Probably with extra overhead as if this file is about to be displayed. Then its contents were scanned for matches as if it was regular search. Simple, elegant and SLOWWWW.

The #1 required performance optimization is to have “search files as utf-8” feature which is basically grep. Assume all files are utf-8 (or ascii or binary). Scan them with efficient buffer by buffer linear algorithm without loading the entire file to Notepad++. But this is a different new feature, not optimization of current search which is still needed by people who (for some reason) use multiple encodings.

The #2 performance optimization is probably asynchronous I/O before multithreading. Maybe with modern NVME drives this is less true than it used to be with spinning disks. Still true for search over network share.

Multithreading is 3rd priority. It is extremely unlikely with original algorithm and rather easy for search as utf-8.

But again, opinions of people who can’t mix and match the right tool for the job and insist that every tool should adapt itself to their selfish needs should not have much weight.

guy038

Hello, @gstavi and All,

I completely agree with both of your statements:

Professionals, when they reach that scale (and long before), use command line tools. Then they open the found files in an editor. Possibly Notepad++.

And :

But again, opinions of people who can’t mix and match the right tool for the job and insist that every tool should adapt itself to their selfish needs should not have much weight.

Best Regards,

guy038