But note that being able to search for "carriage return", "new line" and "page break" (and possibly also the other ones you mention) depends on those being present in the internal representation of text. I am not sure at all these *typographical concepts" exist in the internal representation of text in OpenOffice.org/LibreOffice. I think I have been told that OOo/LO uses a much more "structured" approach with separate objects for paragraphs etc, and maybe then even stores forced line breaks just as data structures, not as actual embedded carriage returns and/or new line characters.

So implementing this might be much more complex than what it perhaps is in MS Office. That doesn't mean it wouldn't be useful, of course. Even if we keep the traditional OOo way to store text in LibreOffice, we could present to the user an illusion that also the formatting characters you mention are actually present. That might be useful for people migrating from MS Office.

On the other hand, for the (few...) people who actually prefer to think of documents in a structured fashion and not as stream of characters including formatting characters, being able to search for for instance carriage returns sure would seem unnatural. In an ideal world, that is how one should conceptualize documents, no?

Of course, I might be totally misunderstanding stuff above, and in that case, feel free to correct me, and/or ignore my rambling.

Hi Tor, 

Thank you for your comments. I think you've indeed understood what I was on about, but:

I understand what you're talking about as regards LibO/OOo's internal storage.

However, that is invisible to the end user.

I, the dumb end user, pressed the carriage return key while typing. I don't care how the software stores it. But I want to be able to search for that carriage return after.

Same thing when I press Shift-Enter (a "new line" or "soft return"). I want to be able to find those "new line" "characters" after.

Same thing for tab "characters". Etc.

Since I made those keystrokes and they have a result on-screen, they are obviously being stored in some form or other. Otherwise, next time I open it, my doc would look different from the way it looked when I typed it, no? ;-)

I sometimes need to search for the "new line" characters and replace them with a "carriage return" and thus create new paragpaphs, etc. Or I need to search for 8 space characters and replace them with a "tab mark" instead.

In MS Office, I have a dropdown list of such "special characters" and it makes life very simple to use them in find/replaces.

Could we get that in LibO, too, please?

Thanks if so. ;-)

Please let me know if I haven't explained clearly. :-)

*** Bug 31509 has been marked as a duplicate of this bug. ***

Please note that the term I meant was NON-PRINTING CHARACTERS, not "special characters"...

(In reply to comment #3)
> *** Bug 31509 has been marked as a duplicate of this bug. ***

(In reply to comment #2)
> Hi Tor, 
> 
> Thank you for your comments. I think you've indeed understood what I was on
> about, but:
> 
> I understand what you're talking about as regards LibO/OOo's internal storage.
> 
> However, that is invisible to the end user.

Indeed, unlike plain text, where you actually can search for and replace newlines (LF), carriage returns (CR) or combinations (CRLF) if you use the right text handling tools.

Some Unicode pointers:
 LF:    Line Feed, U+000A
 CR:    Carriage Return, U+000D
 CR+LF: CR (U+000D) followed by LF (U+000A)
 NEL:   Next Line, U+0085
 LS:    Line Separator, U+2028
 PS:    Paragraph Separator, U+2029

(I wonder how LibreOffice handles plain text files internally, since those characters really *are* there then...)

> I, the dumb end user, pressed the carriage return key while typing. I don't
> care how the software stores it. But I want to be able to search for that
> carriage return after.
> 
> Same thing when I press Shift-Enter (a "new line" or "soft return"). I want to
> be able to find those "new line" "characters" after.

I can't see why LibreOffice couldn't handle these things by allowing the user an easy way to *both* search *and* replace arbitrary combinations of CR and LF, by handling these things inside the content.xml.

> Since I made those keystrokes and they have a result on-screen, they are
> obviously being stored in some form or other. Otherwise, next time I open it,
> my doc would look different from the way it looked when I typed it, no? ;-)


This is what it can look like:
"<text:p text:style-name="Standard">Two paragraphs starting with this line ending</text:p>
<text:p text:style-name="Standard"/>
-<text:p text:style-name="Standard">Two newlines starting with this line ending<text:line-break/>
<text:line-break/>"

It looks like there may be a few cases to handle. Paragraphs seem to have and opening tag (<text:p text:style-name="Standard"/>), and a closing tag (</text:p>) only if there was text in the line, while newlines only have closing tags (<text:line-break/>).

Writing a textutils script that can handle this simple example is a bit of work, but surely not too hard, even for a non-programmer like me, so a pro like the LibreOffice developers shouldn't find it hard at all ;). 

The only potential problem I can see in this simple example, is the "Standard" style-name bit inside the opening tag. Is there a policy for this in LibreOffice, like using the closest preceding one, or polling a standard template?

My guess at why LibreOffice handles it this way, is that it helps make it handle paragraphs, newlines etc. in a uniform way across platforms that have different ways of handling new lines.

> I sometimes need to search for the "new line" characters and replace them with
> a "carriage return" and thus create new paragpaphs, etc. Or I need to search
> for 8 space characters and replace them with a "tab mark" instead.

You're not alone in this. It's a showstopper for me too and many others, reducing LibreOffice to a very limited number of tasks, forcing me to keep MS Office, which I want to get rid of.

In 3.5.1 not implemented yet
> - carriage return
> - new line
> - tab mark
IMHO it is more easy to add to context Help and tooltips information how to search for these characters using regular expressions than actually implement them.
Similarly for replacing for them.

Problem is with this:
> - page break
> - non-breaking spaces
I do not know how to find them using regular expressions.

Implementing graphical user interface (drop-down list) for at least the existing regular expressions, such as \t, \n, $, ^, would be useful to novice users.

There is an add-on ("alternative find and replace" [1]) which does the job, (including probable workarounds of the way LibreOffice stores text? it can actually handle \n in a way different from what the regular expressions page [2] says); it can probably be helpful to implement this bug.

[1] http://extensions.openoffice.org/en/project/AltSearch
[2] http://help.libreoffice.org/Common/List_of_Regular_Expressions

in 4.3.1.2 not implemented yet

*** Bug 87645 has been marked as a duplicate of this bug. ***

*** Bug 137219 has been marked as a duplicate of this bug. ***

Author of dupe bug 137219 here...

I believe this bug mixes up several issues - because the MS Word feature it refers to mixes up these issues:

1. The ability to search for multi-line / multi-paragraph patterns
2. The ability to search for non-printing characters which relate to LO document structure, e.g. LF, CR, PS (see comment #5) and maybe others.
3. The UI for exposing these abilities, and whether it needs to be similar to MS-Word's in referring to "low-level" concepts like LF and CR and their effect, or only to "high-level" concepts like lines and paragraphs.
4. The ability to search for characters which don't relate to document structure, but which can't easily be entered using the keyboard, like non-breaking space and tab, or non-printing characters such as ZWJ, RLM, LRM and so on.

I therefore suggest that these issues be split up into separate bugs with appropriate dependencies/relations between them.

Why?

* NBSP and tab searching are really just a UI shortcoming, they can be searched even now if you paste these characters from elsewhere and there's no reason to hold that up for the other, deeper, more complex issues.
* Part of what can be / needs to be done is allowing for regular expressions like "foo\n.*bar" to work in some reasonable way. That can be done without deciding whether LFs/CRs are exposed to the user or not.


I would like to ask for the CC list members' opinions on this suggestion.

Created attachment 169902 [details]
Simple example of a tiny bit of text with formating.

Created attachment 169903 [details]
Dump of the internal SwNodes structure

Just to give you an idea, how Writer internally sees this simple document. There is simply no way to search for newlines with an regexp. NBSP OTOH has a direct representation in utf-8 (C2 A0) / unicode, that's why you can search for it.

Obviously it's not an unsolvable problem, but nobody found it yet important enough to implement, or even got budget for it.

(In reply to Jan-Marek Glogowski from comment #24)
> There is simply no way to search for newlines with an regexp.

I believe you're misunderstanding... the requested feature is not applying a regexp to the serialization of LO's representation of the document, but rather to implement a regexp in a way in which a more complex search (e.g. using XPath or what-not) would perform a close equivalent of what a regexp application would in a pure textual document.

Specifically, in your example document, the regexp /Two\nThree/ would find a match, starting before the T of Two and ending after the Three, on the next line.

It is up to LO to make this "magic" happen.

(In reply to Eyal Rozenberg from comment #25)
> (In reply to Jan-Marek Glogowski from comment #24)
> > There is simply no way to search for newlines with an regexp.
> 
> Specifically, in your example document, the regexp /Two\nThree/ would find a
> match, starting before the T of Two and ending after the Three, on the next
> line.

Just for this match to happen, you would need to convert the internal representation of newline to "\n", so a regexp can match, and somehow convert a result back. And IMHO that would be really non-trivial. You don't want to write a regexp abstraction over LO internal representation. That would probably be even harder.

(In reply to Jan-Marek Glogowski from comment #26)
> Just for this match to happen, you would need to convert the internal
> representation of newline to "\n", so a regexp can match

You would likely not want run a regexp search. Perhaps something like an XQuery or XPath lookup. i.e. you would probably transform the regex, not the document.


> And IMHO that would be really non-trivial.

Yes, of course it would be non-trivial - it would require quite a bit of programming work. And that is another reason why I think this bug should be split up.