Other thoughts regarding the "ReWrap Text to (Clipboard or 72) width" of the old "TextFX" plugin

guy038

Hello, All,

Quite by chance, I came across a summary of the features of the old TextFX plugin, and among them, I took another look at the Rewrap feature.

First, note that the rewrap feature is generally VERY difficult to simulate as it heavily depends on current text :

How do we manage tables, sections, lists with bullets or numbers and possible leading indentation of the folded lines ?

Or do we need to join all lines of a text in a single very looooooooong line FIRST, before the wrapping process ?

This topic has been discussed many times. see, for example, the following posts below :

https://community.notepad-plus-plus.org/post/62750, from @peterjones

https://community.notepad-plus-plus.org/post/54147 from mine, where I proposed a generic regex

https://community.notepad-plus-plus.org/post/78262 from @ekopalypse, where he proposed a Python script, created with the @peterjones’s help

https://community.notepad-plus-plus.org/post/58881 from @michael-vincent, where he proposed a NppExec script

If we look at it from the perspective of the old TextFX plugin, it would primarily handle text encoded in ANSI. in other words, text where one char = one byte, for EACH character of the selected text !

So, before the wrapping action, we need to :

Use the Edit > Blank Operations > TAB to Space option, in order to replace any tabulation with the corresponding amount of space chars
Use the Edit > EOL Conversion > Unix (LF) option. However, I don’t think that a single line-ending is really necessary
Use the Encoding > Convert to ANSI option to get an ANSI text

Note that the conversion to ANSI could be a problem if it contains some UNICODE characters, absent from the present ANSI encoding. Luckily, the main characters are both present in the different ANSI encodings and in the Unicode encodings, like UTF-8 ! Anyway, all the few non-decoded characters are generally shown as an interrogation mark, within your converted ANSI text !

Now, you may use the Edit > Line Operations > Join Lines option, first, ONLY IF you would like a strict wraping process throughout all the selected text
Then, you run the wrapping process which can be, either, regex replacement(s) or a Python , Lua or NppExec script

So, I created an ANSI file composed of the license.txt, followed with the change.log file of the N++ v8.9.8 release, named ORG_LF_ANSI.txt

Then I copied this file in some new tabs to test the different scenarios !

In this post https://community.notepad-plus-plus.org/post/54147, I proposed this generic regex replacement :

SEARCH (?-s)^(?=.{C})(?:.{1,C-3}\x20|.{C-1})\K

REPLACE \r\n ( or \n if Unix files )

Which mimics the Edit > Line Operations > Split Lines option

However, it happens that, in last N++ releases, this regex replacement should be, rather :

SEARCH (?-s)^(?=.{C})(?:.{1,C-2}\x20|.{C-1})\K

REPLACE \r\n ( or \n if Unix files )

If I choose the value 78 in the Settings > Preferences > Margins/Boder/Edge > Vertical Edge settings dialog, select all the text and use the Ctrl + I shortcut, the file is strictly identical to the other file using the regex (?-s)^(?=.{78})(?:.{1,76}\x20|.{77})\K with \n as replacement ( Verified with ComparePlus v3.00 )

Now, it seemed that in these identical files, in many lines the wrapping process is done before the column 78, even considering a correct wrapping at end of a word !

So, I imagined this new regex replacement, below :

SEARCH (?-s)^(.{1,C})(?:[\x20\r\n]*\z|\x20*\R|\x20+) ( with C = 78 in our case )

REPLACE \1\r\n if Windows file \1\n if Unix file \1\r if Mac file

Finally, run an Edit > Blank Operations > Trim Trailing Space operation ( or SEARCH \x20+$ and REPLACE with Nothing )

And indeed, the New_LF_ANSI.txt file better respects the 78 limit than the Old_LF_ANSI.txt , as shown below :

BEFORE :

And

AFTER :

And

I also created a New_CRLF_UTF-8.txt file, using the last (?-s)^(.{1,78})(?:[\x20\r\n]*\z|\x20*\R|\x20+) regex and the only differences with the New_LF_ANSI.txt file are :

Because the two chars “ and ” are coded with one byte ( 93 and 94 ) in the ANSI text and coded with three bytes ( E2 80 9C and E2 80 9D ) in the UTF-8 text

Now, the wrapping process of both the TextFX plugin and the Python script are quite different : Here is, successively, a screenshot from the TextFX_ANSI.txt file and from the PythonCRLF_UTF-8.txt file :

You can see, in the change.log section, that lines regarding, obviously, different points are badly concatened in a single line !

Now, if I compare these two files, ignoring all spaces and line-endings, ComparePlus v3.00 returns the picture, below, with differences mostly at end of the license.txt section and in the change.log section !

Finally, I created the NEW_LF_ANSI_Strict.txt file :

I did, first, an Edit > Line Operations > Join Lines operation which changed all text in a very long line ( 37,410 chars )
Then, I applied the following regex replacement against this file
- SEARCH (?-s)^(.{1,78})(?:[\x20\r\n]*\z|\x20*\R|\x20+)
- REPLACE \1\n

Here is a screenshot of the resulting file, showing the end of the license.txt section and the beginning of the change.log section

As you can verify, the reading of this text is rather painful :-((

Best Regards,

guy038

mpheath

@guy038 Seems like regex Wrap versus the others using ReWrap. The test does not appear to be an equal comparison.

The test input files are UnWrapped. ReWrap would probably be for lines that are already Wrapped. Neither of the test files have a fully strict form of paragraphs. The license file is mostly paragraphs except for some lines.

GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007

These 2 lines are supposed to remain as 2 lines, not 1 paragraph. So if treated with ReWrap, would probably result as 1 line.

Wrap can be done with this Lua code:

local function main()
    local maxLineLength = 72
    local line = 0
    local eol = ({'\r\n', '\r', '\n'})[editor.EOLMode + 1]

    editor:BeginUndoAction()

    while 1 do
        local lineLength = editor:LineLength(line)

        if lineLength < 1 then
            break
        end

        if lineLength > maxLineLength then
            local startPos = editor:PositionFromLine(line)
            local endPos = editor.LineEndPosition[line]
            local maxPos = startPos + maxLineLength
            local breakPos

            for i = startPos, endPos do
                local posAfter = editor:PositionAfter(i)

                if posAfter > (i + 1) then
                    maxPos = maxPos + posAfter - i - 1
                end

                if editor.CharAt[i] == 0x20 then
                    if breakPos and i > maxPos then
                        break
                    end

                    breakPos = i
                end
            end

            if breakPos and breakPos < endPos then
                editor:SetTargetRange(breakPos, breakPos + 1)
                editor:ReplaceTarget(eol)
            end
        end

        line = line + 1
    end

    editor:EndUndoAction()
end


main()

It replaces the space <= 72+ to break at with an EOL. Suggest to keep the original unwrapped copy as I have no UnWrap or ReWrap code.

Coises

This post is deleted!

Coises

@guy038 said:

Quite by chance, I came across a summary of the features of the old TextFX plugin, and among them, I took another look at the Rewrap feature.

First, note that the rewrap feature is generally VERY difficult to simulate as it heavily depends on current text :

How do we manage tables, sections, lists with bullets or numbers and possible leading indentation of the folded lines ?

Or do we need to join all lines of a text in a single very looooooooong line FIRST, before the wrapping process ?

As far as I can tell, the old TextFX didn’t handle any of the more complex cases (tables, lists, indentation). I have not been able to find user documentation. The closest reasonably accessible code I could find was this for the rewrap function, which has comments suggesting some fancier processing might be possible, but it doesn’t implement it.

Do any of the scripts or proposals offered so far do anything that can’t be accomplished with the native Split Lines function? Split Lines splits at the edge of the view if you have no vertical edges set, and at the last (or only) vertical edge listed if you do. (So if you list 72 80 it will split at 80; if you list 80 72 it will split at 72.)

What is the problem to be solved?