[New Plugin] CSV Lint

Thomas McArthur

@Bas-de-Reuver I found the cause of the crash: it’s the Multi-Line setting for the Tab Bar. When that is enabled, and the tabs wrap to 2+ lines, activating a CSV tab a 2nd time crashes NP++.

Enable multi-line, and only have one line of tabs - no crash.
Un-maximize the window, drag it small enough to create 2+ rows of tabs - activating a CSV tab a 2nd time crashes NP++.

This crash even happens on a fresh download of NP++ v8.5.2 portable.

rdipardo

@Thomas-McArthur, I can’t make it crash for me, but it could simply be I don’t understand your instructions.

Does “Multi-Line” mean that long lines are wrapped, i.e. the option at View > Word wrap has a ✅ beside it?
Exactly how many different files are open at the same time? Is there a CSV file, and one more file of a different type? Or is the same CSV file open in multiple views, i.e., did you go to View > Move/Clone Current Document and then click Move/Clone to Other View?

Snabel42

@rdipardo I think this is checked: Settings / General / Tab Bar / Multi-line

Then make sure the window is narrow enough where the Tab bar has more than one line.

edit I can not replicate in version 0.4.6.4 64-bit

Chr Pqe

This plugin is a gamechanger. There are few bugs here and there nevertheless. For example,

it will not properly detect documents in the second NPP pane.
it seems to crash NPP a lot with CSV files using a lot of white or blank lines (maybe 50-100+ white/blank/tab lines at random positions)

I understand that this not a CSV or spreadsheet editor or a data analysis plugin, but some obvious features are missing :

Change column position - add a simple menu item or button (in the CSV Lint window) to swap columns of the CSV
Sort by this column - this is an obvious feature that is missing, nevertheless for this one, if the first feature is implemented (change column position), I could simply swap the needed column to the first one then use NPP sort and it should do the same thing.

I don’t think “changing column position” should be hard to integrate, since every column is already detected by a color (thus position).

Also, the reason it is a bit troublesome to not have this feature, is that a lot of times CSV columns are not in the correct position to visualize and understand the data, which forces to open Excel or some spreadsheet just to modify the position of the column and go back to NPP.

Chr Pqe

@artyb55 In your case for example, having a “change column position” would be useful aka you insert your column at the begening, then swap/change position of this newly edited column at the place you want

Coises

@Chr-Pqe said in [New Plugin] CSV Lint:

I understand that this not a CSV or spreadsheet editor or a data analysis plugin, but some obvious features are missing :

Change column position - add a simple menu item or button (in the CSV Lint window) to swap columns of the CSV

Sort by this column - this is an obvious feature that is missing, nevertheless for this one, if the first feature is implemented (change column position), I could simply swap the needed column to the first one then use NPP sort and it should do the same thing.

It is possible that a plugin I’m currently developing, Columns++, would help you, though there are some significant caveats:

It’s in alpha stage right now. I can’t promise it won’t crash Notepad++, so it’s important to keep work safe.
You would need to use its conversion functions to convert separated values to tabbed display, then convert back if you need to save the document in its original form. Depending on the details of your file, some care with the conversion options may be necessary if you need it to “round trip” unchanged except where you’ve made edits.
You would need to enable its Elastic tabstops feature to line up the columns; then you could use ordinary rectangular selections to move columns by cutting and pasting (admittedly, still a somewhat clumsy operation), and there are sort commands in the plugin that work with tabbed documents. Unfortunately, performance of Elastic tabstops is quite dependent on file size; e.g., I can edit a file with 1300 lines and 12 tabs per line and rarely notice any impact, but with 19,000 lines and 50 tabs per line, some operations can freeze for half a minute or more.

There’s a thread over here about it, so this one doesn’t get cluttered, if you have questions, comments or reports.

Bas de Reuver

@Chr-Pqe Thanks, nice to hear you like the plugin. 👍 Are there any functions of the plug-in that are especially helpful? I’d like to know because I can’t see which functions are most used or which aren’t used at all.

A sort function was in fact already added a while ago in v0.4.6, also see comment of 9 november 2022 in this thread, which version of the plug-in do you have installed? When you open the CSV Lint docking window Plugins > CSV Lint > CSV Lint window there should be a Sort button. It even sorts date/float/integer columns correctly according to their values.

And I agree, being able to change the column order would be a nice feature to have.

Bas de Reuver

@Chr-Pqe You say the plugin sometimes crashes, does that happen just when opening a csv file or when you do Reformat or something like that?

At the moment there is a known bug with the transparant cursor line, see issue #67 thanks to @Thomas-McArthur for posting it.

Does it still crash when you set the CSV Lint setting TransparantCursor to false?

Bas de Reuver

@Alan-Kilborn @rdipardo If possible, could you take a look at this issue #67 ? I’m 100% certain that this has to do with these two lines i.e. when I disabled them then it doesn’t crash anymore. However at the moment I can’t figure out why that is.

Btw the lines were introduced in this commit and also see this post

Alan Kilborn

@Bas-de-Reuver

It appears that @rdipardo had some thoughts on it, HERE.

Bas de Reuver

The CSV Lint plug-in was updated to v0.4.6.5. The biggest new feature is enumeration (coded values) meaning the plug-in now also tries to detect column with coded values. The big advantage is that you can now also check if a coded column contains invalid values, so values outside of the provided codelist. Other than that it’s mostly minor updates and bugfixes

Support for enum/coded values
Analyse Data Report, show correct date range
Fix rare crashing situation due to transparent cursor line setting
Syntax highlighting fix for special separator character
Convert to SQL, PostgreSQL syntax improvements, support enum and table comment
Generate Python/R script, support enum
Generate W3C CSV Schema JSON various fixes and improvements
Settings dialog, hide userpref dialog values
Various updates and bugfixes

antoineDion

@Bas-de-Reuver said in [New Plugin] CSV Lint:

Allow user to delete column

I installed version 0.4.6.5 and I can’t find how to delete a column.

Bas de Reuver

@antoineDion said in [New Plugin] CSV Lint:

@Bas-de-Reuver said in [New Plugin] CSV Lint:
I installed version 0.4.6.5 and I can’t find how to delete a column.

It’s a bit of a hidden feature, located at an unintuitive menu option in the plugin. You can go to “Add column” (this is the unintuitive part) and then select the original column you want to remove, then select none of the options by right-clicking (this possibility was added in 0.4.6.5) and then check the “Remove original column” checkbox.

Also see the documentation here

Note that you can also right-click the radio buttons to deselect them. By not selecting any of the options and checking the “Remove original column” you can press OK to simply remove the selected original column.

Sebastijan Stanivuk

I have this issue when I try to Validate csv “is not a valid enumeration member” what does it mean?

** error line 6: Column 5 value "Tall car with medium long brown hood. Maybe English, special about it was it had windows like the bottom of Coca-Cola. Nice car, not very cute. Taller then my." is not a valid enumeration member,

Mark Olson

@Sebastijan-Stanivuk
X is not a valid enumeration member means that CSVLint thought that a certain column of the CSV would contain only certain values, but X was not one of those values.

nousername1

Hi
I work often with Open Data who are formatted sshhttt often. Is there some good how to to make the best out of the Plugin?
Is there some methode to Automatise a Script to even fine tune some Parameter afterwarts and play them when a new File comeout again and again…?
Thanks

Bas de Reuver

@Sebastijan-Stanivuk The enumaration/coded values is a relatively new feature, but the automatic detection was too strict, see also this issue.

In practice it now incorrectly flags a column with comments/text that has very few non-empty values as “coded value column” and then list all non-empty texts as the coded values, even if they each appear only one time.

In the next CSV Lint version will have improved enumeration detection, i.e. values in a coded column must appear at least 2 times or more on average before it’s actually flagged as “coded value”.

Bas de Reuver

@nousername1 I think you’re asking if the CSV Lint menu items and buttons can be called automatically, is that correct? So that you can check or validate many CSV files automatically?

It’s an interesting question, but no that’s not possible at the moment, the plugin doesn’t provide access or an API or something like that, i.e. if you want to validate 5 csv files you have to do it interactively in the Notepad++ GUI so by hand.

If you want to automatically check many files or convert data, then the only alternative I see is to create a separate script in Python or R or Powershell or something like that.

Bas de Reuver

The CSV Lint plug-in was updated to v0.4.6.7. The biggest new features are a fix for a longtime unnoticed bug in the Reformat/Sort/Add Column functions, and improved enumeration detection (fewer false positives, previously text columns with very few values would always be flagged as codes, which was incorrect). Also some improvements in generating the PowerShell script, a new Sort-on-length option and several small fixes that needed to be released.

Reformat, Sort and Add Column changes CRLF to LF
Improved enumeration detection
Sort dialog, new option Sort on length of value
Sort and Add Column, allow resize of dialogs for long column names
Generate scripts, various improvements
Various bugfixes

Florian Hoellwarth

When using the background colour highlighting mode, would it be possible to have different (darker?) background colours in the line currently selected?

Right now, CSV Lint seems to be overridden by the “Highlight Background” Current Line Indicator of Notepad++, see first line below:

EDIT 1: This seems to be an issue in my configuration, the screenshot in https://community.notepad-plus-plus.org/post/81307 shows the highlights just fine:
alt text

EDIT 2: After setting the “Transparent Cursor” option to FALSE, restarting Notepad+, setting it back to TRUE again, everything looked the way it should:

After restarting, though, the whole line was only gray again.

I’m not sure, but I think this has to do with issue 15035 in Notepad++ Github (I’m not allowed to post links yet).

–
moderator added link