Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.

Terry R

@Black-Fox said in Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.:

The AFTER is rather simple as you see. I hope you got the point…

Regular expressions are going to most likely transform most of the lines you seek to change. However I’m a bit unsure of the very first requirement. You say “everything from line 274 and above to be deleted” yet you also appear to say line 274 is not fixed.

Now the 2nd requirement is working on successive lines (275, 276, 277 etc). That appears to be contrary to the 1st requirement. So I expect that your 1st requirement was actually to say you need lines 1-274 to be deleted (for me 274 and above means 274, 275 etc to the end of the file). You also say that the #274 may be different depending on the file. If it can be different you would need to provide the method by which you determine the line to stop deleting. I expect that may well be something along the lines of your 2nd requirement, thus the line would contain “<InstrumentSetup id=”.

So I would suggest you read the FAQ post called Template for Search/Replace Questions here. You need to insert examples of the before and after within code blocks as otherwise some characters may well be interpreted/altered by the posting engine, quotes " being one of them.

Terry

mkupper

@Black-Fox said in Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.:

i) In the Before file, i want everything from line 274 and above to be deleted. (the line 274 is not fixed in every different file).

That part is an easy low hanging fruit:

Search: (?s-i)\A.*</Equipment>$
Replace: (nothing or blank)

Breaking that down:

(?s-i) The s switches on the dot matches newline flag so that the .* later in the expression will scan past the end of lines on to the next line. The -i part turns off the ignore-case flag so that we end up scanning for Equipment with the upper case E and lower case quipment.
\A Says to start at the beginning or line 1 of the file.
.* Matches any and all characters until …
The match stops at </Equipment>$. If you also want to remove all of the line with the </Equipment> on it then use </Equipment>\R instead of </Equipment>$ as \R matches the end of line mark while $ matches up to but not including the end of line mark.

Terry R

@Black-Fox said in Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.:

Could you please help me with the following search-and-replace problem I am having?

I’ve taken a different approach to @mkupper in that I target the lines you do want to keep as I was unsure of whether other tags such as “Equipment” might be embedded within the area you did want to keep.
So, using the Mark function I would use:
Find What:(?s)<InstrumentSetup id=.+
Have “Bookmark line” ticked (search mode must be Regular Expression) and then click on the “Mark All” button. Now that the lines to keep are marked you can remove the “unmarked lines” by using the menu option Search, Bookmark, Remove Non-Bookmarked Lines". As long as you had the cursor at the start of the open file when running this Mark function, this will find the first occurrence of “<InstrumentSetup id=” and mark all lines from here to the end of the file.

As I said before I believe all your requirements can be met by a number of regular expressions. Then these can all be stored together in a macro. This would mean all you need to do is open a file, then run the macro and if everything you tell us is accurate enough you should have the result you need.

As for all the other line changes, you do need to provide them in the code boxes. Currently you have so many requests presented it’s hard to understand everything as you currently have them presented. Some of your requests include statements like “Then we go to Line 743 of Before where we repeat step ii (line105 of After), Line 748 and 749 where we repeat step iii (line106 of After), then we go to Line 751 and 752 of Before where we repeat step iv (line 107 and 108 of Before)”. So the question is: Are there occurrences of the lines which you don’t want altered. From what I can see in your “after” example it does not appear to be so.

So if EVERY line that has a particular string in it that will need altering, telling us where that line might be; “line 743 of Before”; is redundant. All you need to say is “after the initial deletion of lines 1 to approximately 274” where ever “this string” occurs, it needs to change to “this format”.

Terry

Black Fox

Thank you all in advance.
Indeed i want to delete everything from 1 to 274. But that is the least of my problems, as it takes less than a sec to do it by myself.
Everything else is my difficulty. And unfortunately i don’t know the expressions to filter what i need and replace for an easy outcome.
The steps i to vi is a repeating process of the data inside the before file. What i mean is : Lines 275 to 742 is one period. Lines 743 to 1254 another period, etc. Each period has the same structure, Just the names change. The gap between lines 318 and 723 contains the same “paragraphs or sections” as the lines 309-317 so i didn’t repeat them more.

As a conclusion, what i need is :

Example: The lines 283 and 284

283<RawObservation setupID=“TPSSetupID_S1_7” purpose=“backsight” targetHeight=“1.765000” horizAngle=“315.975684” zenithAngle=“87.048171” horizDistance=“18.538283” slopeDistance=“18.928679” directFace=“true” timeStamp=“2024-10-17T12:15:54”>
284<TargetPoint name=“S2” pntRef=“S2@9”>1004.603425 982.042372 103.549492</TargetPoint>

to be altered to this:
BS S2,1.765,
SD 315.975684,87.048171,18.928679

Can you help me with the expressions?

Terry R

@Black-Fox said in Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.:

Can you help me with the expressions?

Now you still haven’t been able to show us exactly how you know to replace “<RawObservation”… with "BS. I think I see “S2” later in the 2nd (#284) original line, but you also have a “SD” which like “BS” seems to appear from nowhere.

And still you persist in showing examples without the code boxes. Are the numbers 283 and 284 actually part of the file? So as I see it the example should have actually been presented like this:

<RawObservation setupID=“TPSSetupID_S1_7” purpose=“backsight” targetHeight=“1.765000” horizAngle=“315.975684” zenithAngle=“87.048171” horizDistance=“18.538283” slopeDistance=“18.928679” directFace=“true” timeStamp=“2024-10-17T12:15:54”>
<TargetPoint name=“S2” pntRef=“S2@9”>1004.603425 982.042372 103.549492</TargetPoint>

and

BS S2,1.765,
SD 315.975684,87.048171,18.928679

assuming the 2 old lines changes to 2 new lines (transposing data). All I can understand so far is it seems you must use both old lines (283 & 284) to produce the new lines, moving strings between them.

These code boxes are explained in the link I gave you previously. In short, you insert the original data (without anything added or subtracted, so no line numbers if they weren’t in the original file), then select that data and click on the </> button above the post as you are creating it.

Terry

Terry R

@Black-Fox said in Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.:

As a conclusion, what i need is :

Example: The lines 283 and 284

From that example I would consider this may be the change you want to make:

For every line starting with “<RawObservation”, use this and the next line to make 2 new lines.
The first new line will start with "BS " followed by the code that appears in the quotes in “name=""” in the following line, then a comma followed by the “targetHeight” number in the first line, again followed by a comma.
As shown in the example, any number such as shown (1.765000) should be shown without any following zeros), so shown as “1.765”.
The 2nd new line will start with "SD ", followed by the number for horizAngle, a comma, then the number for zenithAngle, a comma and finally the number for horizDistance.

Now, from your first post it would seem that there are lots of different transformations you need to make. If they can all be represented as I’ve shown then it certainly is possible to provide you with regular expressions to make the changes.

It now lies with you to present the examples in code boxes, before and after in seperate boxes and to also present the transformations in a similar fashion to how I explained it. Unless you can be clear in your own mind and present the requests clearly you are going to be waiting a long time, and many back and forward posts such as this before you get a solution. It does seem that you need to solve this quicker rather than slower.

Terry

Mark Olson

Your problem is kind of on the upper limit of reasonable things to automate in a macro; you could do that, but you almost certainly would be happier with a script written in a scripting language (and by this I mean a script outside of NPP, not something like a PythonScript script).

Terry R

@Mark-Olson said in Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.:

and by this I mean a script outside of NPP, not something like a PythonScript script

Very good point. Actually every time I re-read the OP I get more info, but it’s so cluttered that I am getting to the point where I wonder if a regex solution would be robust enough.

Question to @Black-Fox:
This is very likely something that may have been achieved before. In all likelihood, the manufacturer of either your old equipment or new equipment may have produced a translation program to do exactly this. And if available, as a commercial product (yes you may have to pay for it) it will “guarantee” a correctly transformed file.

Regular Expressions will not guarantee that. Every effort is made by us (the solution providers) but we give no guarantee!

Terry

Black Fox

@Terry-R

I figured so that it would be to complicated to do this alone.
I actually have asked the software manager to help me, but he hasn’t done it yet (and i am waiting months) and everytime i am doing this all by myself manually…
As for the numbers, yes they are not part of the code, but i put them so to explain which line of Before corresponds to which line of After.

Can i ask you one last thing? Can you write me 2 “expressions”?

I want this:

To be replaced with this :

94.912866,81.131542,17.821968

and i want to select each of this so to be erased (the numbers are variables) :

” pntRef=“2@11”>1001.360583 1016.990502 106.695188</TargetPoint>
<Feature code=“observationInfo”>
<Property label=“TPSCorrectionRef” value=“3”/>
<Property label=“EDMType” value=“reflectorless”/>
<Property label=“reflectorName” value=“Reflectorless”/>
<Property label=“reflectorConstant” value=“0.034400”/>
</Feature>
</RawObservation>

Again, thank you all. I’m so pressed by the time for this work and is a huge waste of my time doing it for each line…

mkupper

@Black-Fox said in Search and replace Question - From LEICA.xml format to TOPCON.txt format for surveying.:

<RawObservation setupID=“TPSSetupID_S1_7” purpose=“normal” targetHeight=“0.000000” horizAngle=“94.912866” zenithAngle=“81.131542” horizDistance=“17.044892” slopeDistance=“17.821968” directFace=“true” timeStamp=“2024-10-17T12:17:29”>

To be replaced with this :

94.912866,81.131542,17.821968

First off, it would help greatly if you use the button that’s in the toolbar for foryum messages when posting the text of your data or proposed regular expressions. The forum software tends to mangle text to make it “prettier” for humans meaning what others can copy/paste from the forum for testing is not exactly what’s in your data. Just select some text and click the button. It will wrap the text in triple single back tick things. These are a clue to the forum software that it must not “prettify” or otherwise mangle the text.

As you did not use the magic button I’ll need to guess that you intended to post that

<RawObservation setupID="TPSSetupID_S1_7" purpose="normal" targetHeight="0.000000" horizAngle="94.912866" zenithAngle="81.131542" horizDistance="17.044892" slopeDistance="17.821968" directFace="true" timeStamp="2024-10-17T12:17:29">

should be transformed into

94.912866,81.131542,17.821968

I would do this one of two ways:
Search: (?-i)<RawObservation setupID="[^"]*" purpose="normal" targetHeight="[0-9]+\.[0-9]+" horizAngle="([0-9]+\.[0-9]+)" zenithAngle="([0-9]+\.[0-9]+)" horizDistance="[0-9]+\.[0-9]+" slopeDistance="([0-9]+\.[0-9]+)" directFace="true" timeStamp="20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]">
Replace: $1,$2,$3

or
Search: (?-i)<RawObservation setupID="[^"]*" purpose="normal" targetHeight="[0-9]+\.[0-9]+" horizAngle="(?'horizAngle'[0-9]+\.[0-9]+)" zenithAngle="(?'zenithAngle'[0-9]+\.[0-9]+)" horizDistance="[0-9]+\.[0-9]+" slopeDistance="(?'slopeDistance'[0-9]+\.[0-9]+)" directFace="true" timeStamp="20[0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]:[0-9][0-9]">
Replace: $+{horizAngle},$+{zenithAngle},$+{slopeDistance}

They are exactly the same expression but for the second version I used group names rather than group numbers. It makes the expressions easier for humans to read and follow.

As there is so much clutter in the previous messages I did not look to see if there are possible exceptions that I should account for in the search/replace I just posted. If the search part does not match 100% of the RawObservation things you have then post the exceptions, using the magic button, and we can help with dealing with those expression.

I usually to a first test where I search for just <RawObservation and hit the count button. I make note of the count. I then write the long regular expression and check that it also matches exactly the same count as the original <RawObservation. If not, I know there are exceptions I need to figure out and deal with.

Black Fox

@mkupper Thank you! Thank you sooo much!
You saved me at least one hour of manually selecting and deleting!!!
Thank you all very much!