[New Plugin] CSV Lint

Alan Kilborn

I was mainly referring to where things seemed to be going with “Here’s the help description for the Column Separator”…

But OK, if this is helpful to CSV Lint plugin…

artyb55

Hi,
I don’t know if this the best place to ask for features for the plugin?

I need to work with CSV files that aren’t compliant with RFC 4180 CSV rules, and I can’t change their format. They have some rows of differing lengths, and some fields are encloses in quotes even though they don’t strictly have to be according to the standard. I’ve been searching for an editor that will allow me to move some fields, and delete some columns, WITHOUT changing the formatting of the CSV file. I don’t know that could be added as an extension?

Or do people have suggestions of other software to use? I’ve tried LOTS of editors and only found one commercial piece of software that does what I need.

BW,
A

PeterJones

@artyb55 said in [New Plugin] CSV Lint:

Hi,
I don’t know if this the best place to ask for features for the plugin?

Best, no: the plugin has a github issues page where you can make feature requests. But the developer of that plugin does swing by here occasionally, and will likely eventually see your request.

I need to work with CSV files that aren’t compliant with RFC 4180 CSV rules, and I can’t change their format. They have some rows of differing lengths, and some fields are encloses in quotes even though they don’t strictly have to be according to the standard.

Neither of those are a problem for the CSV Lint plugin. Well, extra quotes is not a problem at all. The too-few-fields has a workaround in CSV Lint.

If you load a CSV with CSV Lint active, and it highlights it all blue (pretends it’s one column), then do Plugins > CSV Lint > CSV Lint Window, make sure the checkbox for “automatic” is unchecked, and click Detect Columns to manually detect the columns: at that point, it will pop up a dialog:

Tell it you want comma as the separator, and check or uncheck the “header names” option. Then it will highlight based on number of fields per line, even if there aren’t a consistent number of fields.

I’ve been searching for an editor that will allow me to move some fields, and delete some columns, WITHOUT changing the formatting of the CSV file. I don’t know that could be added as an extension?

Well, CSV Lint is not a spreadsheet editor, and I don’t think it has column-moving or column-deleting options. My guess is that adding the change the order of columns would be a huge feature addition. As would “field” operations like “easily” moving from one field column to another for just one record.

Or do people have suggestions of other software to use? I’ve tried LOTS of editors and only found one commercial piece of software that does what I need.

This forum is focused on Notepad++. Notepad++ is focused on editing the text of the file, not the “spreadsheet-like qualities” behind that text. And a Notepad++ text-editor forum seems a strange (off-topic) place to ask for recommendations for non-Notepad++, non-text-editor software recommendations.

But in Notepad++, you might be able to use the “CsvQuery” plugin to use SQL commands to manipulate the data; you might be able to delete a column in SQL commands, or move field values around, and then have that plugin take the final state of the SQL table and put it back as the raw CSV… but I only do SQL for about 1 day per year, and couldn’t tell you how; and I don’t know that plugin well enough to know if it saves the final state of the table as CSV in the same format that it started as (nor what it would do, whether it would remember extra quotes or not).

Lycan Thrope

@Pierre-de-la-Verre ,
Well, now you’re changing a why is this doing this and how do I do it right to this should be improved to do what I want.
If it’s not doing what you want, you should make a feature request to the author of the plugin, the website link of which is in the plugin’s About dialog.
There you can make a request, and a donation, which might even get their attention beyond their just being interested in implementing that kind of feature for their own use.

Thomas McArthur

CSV Lint crashes NP++ using 32bit v8.5 and a 750 line CSV file. I didn’t notice the version of CSV Lint before I uninstalled it, but the plugin updater said there were no updates to install.

Terry R

@Thomas-McArthur

Do you have any other plugins? Sometimes these faults occur because of some side effect from a plugin without seemingly using that particular plugin.

You’d need to recheck and provide the debug info from your installation before anyone can take serious notice.

There is a FAQ post called Notepad++ Crashes/Freezes/Unresponsive after Update which explains what’s necessary to better identify the culprit.

Terry

Thomas McArthur

@Terry-R Done. It still crashes. Here are my specifics:

I uninstalled all other plugins
I reinstalled CSV Lint
I use dark mode in NP++
Select CSVLint in the Language menu
Both dark modes in CSV Lint (Neon and Pastel) crash NP++
I have multiple files/tabs open in NP++.
Tabs are configured for multi-line, no X/close button on tabs, d-c to close
The crash happens with these steps:
8a) Launch NP++
8b) a CSV is active when NP++ launches
8c) Select a non-CSV tab
8d) Reselect the CSV tab that was open when NP++ launched
8e) NP++ crashes with no errors or popups
If I leave None (Normal text) selected in the Language menu, NP++ does not crash, no matter how many times I select the CSV tab

NP++ Debug info:

Notepad++ v8.5 (32-bit)
Build time : Mar 6 2023 - 02:15:26
Path : C:\Program Files (x86)\Notepad++\notepad++.exe
Command Line :
Admin mode : OFF
Local Conf mode : OFF
Cloud Config : OFF
OS Name : Windows 10 Enterprise (64-bit)
OS Version : 22H2
OS Build : 19045.2604
Current ANSI codepage : 1252
Plugins :
CSVLint (0.4.6.2)

artyb55

@PeterJones
Thank you for the helpful reply. That’s what I’ve been doing. The next step to allow me to edit the CSV file in a text editor rather than a spreadsheet type program is to use the ‘Reformat’ and ‘Align vertically’ options in CSV-lint so I can insert some extra delimiters in some rows and then use Notepad++'s column editing features to remove columns. However at that point all the quotes get stripped out of the file, and the options to replace them don’t match the format of these files. I’ll ask on the Issues or Feature Requests page that @Lycan-Thrope highlighted. I had searched for where to ask questions, sorry I didn’t find the right place. I thought that the feature I was looking for might be useful to other people too, and were in the scope of CSV-lint and Notepad++.
BW,
A

Thomas McArthur

@Thomas-McArthur Crash update.

With CSV Lint installed with colorization enabled, if I just open a CSV, then close it, then open another, the colorization works normally, and NP++ does not crash. I can even D&D multiple CSV’s into NP++, and each one opens (in color) without crashing. But as soon as I activate a CSV tab from any other tab, NP++ crashes.

Lycan Thrope

@Thomas-McArthur ,
Just on the off chance that this might have something to do with the issue you’re having, why, if you have a 64 bit system, are you using a 32 bit version of NPP?
Perhaps, there is a problem dealing with the size of page switching memory between the two?
I have no trouble switching my .csv files tab to tab, so at this point, that is about the only thing I see that might be causing your problem. Either that or you have memory or drive corruption if it’s swapping on disk? Just a thought. Try doing the same thing with the 64 bit version of NPP and see if that doesn’t fix the problem.

Pierre de la Verre

@Pierre-de-la-Verre said in [New Plugin] CSV Lint:

Hi
I just started with this plugin and have a problem …
…

Solved here: https://github.com/BdR76/CSVLint/issues/58#issuecomment-1496656520

Short story: Use “Reformat” only with “Align vertically / Re-apply vertically: none” - that’s all

Thomas McArthur

@Lycan-Thrope I use the 32bit version because the plugin support for the 64bit version was extremely limited. Plugins that I used daily weren’t available in 64bit, so I stayed with 32bit. Memory is not an issue - I am using less than half of the 32GB that is installed. I will try the 64bit again to check out the plugin support.

Michael Vincent

@Thomas-McArthur said in [New Plugin] CSV Lint:

Plugins that I used daily weren’t available in 64bit

It’s getting better - are there still some 32-bit only plugins that you can’t part with?

Cheers.

Alan Kilborn

@Thomas-McArthur said in [New Plugin] CSV Lint:

Memory is not an issue - I am using less than half of the 32GB that is installed.

I’m curious what this means.

Bas de Reuver

@Thomas-McArthur and @Lycan-Thrope about the issue of CSV Lint and Notepad++ sometimes crashing when switching between different CSV tabs.

I think this is the same as issue #32, and recently I was able to reproduce a similar situation. That situation was due to potentially using an out-of-sync reference to the ScintillaGateWay and I’ve fixed it in this commit

Can you verify that it doesn’t crash anymore by using the latest development build? The syntax highlighting colors sometimes still don’t refresh immediately, only when you focus the file (click mouse somewhere in csv text), but at least it shouldn’t crash Notepad++ anywmore.

Mark Olson

@Bas-de-Reuver
I used to have problems with multi-instance/multi-view in JsonTools (I use an editor singleton global variable to interface with Scintilla), and what worked for me was to make it so that whenever the plugin receives an NPPN_BUFFERACTIVATED notification, the global editor is reassigned with editor = new ScintillaGateway(PluginBase.GetCurrentScintilla());.

In principle this approach should never fail, because that notification will always fire whenever the user toggles between instances/views. If one of the gurus can think of a reason why I’m wrong, I’d love to hear it.

Granted, you’d have to refactor all your existing code to remove all those calls, but it might simplify matters for you.

Thomas McArthur

@Bas-de-Reuver I tried the 64bit development build, but I get an error that it’s not compatible with v8.5.2. Are there dev builds of NP++? I don’t see any in the downloads page.

However, using your dev build may not be unnecessary because I can reproduce the problem in the non-dev 64bit version. The crash only happens when I have a CSV file that is too wide to fit on the page, so I turn off word wrap. With word wrap enabled, I can switch between CSV and non-CSV tabs fine. But when I enable word wrap in my 380-character wide CSV file, NP++ crashes when switching to that tab (and colorization is enabled).

Mark Olson

@Thomas-McArthur
The only way I know of to get a dev build of Notepad++ is to build it from source using Visual Studio.

I’ve tried CSVLint on my own 8.5.2 dev build and I can toggle word wrap on and off even with 400+ character columns on a 20MB file with no problems.

In my experience, it’s pretty rare for a clean build of Notepad++ to outright crash, so consider checking if any of your other plugins could be causing this.

Thomas McArthur

@Bas-de-Reuver v0.4.6.3 still crashes NP++.

I also tried downgrading NP++ to v8.4.5 and v8.5, and CSV Lint v0.4.6.3 still crashes those versions.

Bas de Reuver

@Thomas-McArthur That does sound like an serious issue, however I can’t reproduce what you’re describing.

I generated a test file with many columns (using generate_data99.py script) and then turning Word wrap on and off while switching between csv/non-csv tabs.

Maybe it has to do with the file encoding, does it also still happen when you first convert the file to a different encoding like ANSI or UTF8 and then save, and then try again? Or possibly it has to do with the session.xml file, does it still happen when you delete/rename your recent %userprofile%\AppData\Roaming\Notepad++\session.xml file and then re-open Notepad++? If it still happens, could you send me an e-mail with your ? > Debug info content?