I started with ShadowHunt-CS-PageFmtNew,Bad-obfusc.odt

1. Deleted all comments to see if some hidden characters were adding space
No change

2. Turned-off Justification (ugh, hate the mess justification makes)
No change (but a lot easier to read)

3. Removed all manual formatting 
Ctrl+A to select all.
Crtl+M to remove manual formatting
That had an effect - more pages now go to the bottom of the page.

4. Looked at Page Style is saw being used - Convert 8
Page size was set to:
- Width 5.00 in
- Height 7.99 in
Changed Height to 8.00 in.
That had an effect - more pages now go to the bottom of the page.

The pages with just text now all seem to go to the bottom.
Chapter pages will still have a gap.
This is due to the chapter title font selected in points and the space before/after paragraph selected in inches will not evenly fit in the page height selected.

You have a mix of point sizes and inches.
The only way to make sure all pages fix exactly into the page size within the margins is to use points and picas everywhere.
That way you can align everything into a grid which fit exactly into the page.
That way you can make the chapter titles a particular point size and set before/after paragraph in point which will make your pages align to say a 12 point grid.

You cannot set a page size in inches, a font size in points, a before/after paragraph spacing in inches - and expect everything to fit.

Also LibreOffice makes rounding errors with font sizes which are not exactly an integer size. I think 10.5, 11.5, etc. does work properly, but other font sizes like 10.3 will not be sized properly. There is a bad bug in LO.

If you are serious about book publishing take the time to learn how to use points and picas.
It will make your life much easier.

That seems pretty strange, but it's useful information if I can figure out what you mean.

I don't understand why you say Page Style (Convert 8) had
Page size set to:
- Width 5.00 in
- Height 7.99 in
Changed Height to 8.00 in.

In my copies, including the files I sent, I have:
- Width 360 pt
- Height 575 pt

None of this explains why the two files, with identical settings for every property of the Convert 8 page style and the CSP Chapter Body paragraph style are laid out differently by LO.  Surely the difference in layout should be reflected by some visible difference in one of the settings?

When you said "You cannot set a page size in inches, a font size in points, a before/after paragraph spacing in inches - and expect everything to fit."

- that's helpful.  But since I have paragraph spacing of 0 for before and after, I assume you mentioned that just for completeness.

I assume the key point you're making is that I'll need to make sure that the Page Height is an integer multiple of all the parts that go to fill the page.  That's really helpful, and I can get to work on that if I know a little more.

Is there a diagram or a wiki page that explains what all the component pieces are?  It's not obvious to me which elements add, and which ones reserve space within some other page element.

I'll also need to know what Leading is used.  Specifically, with spacing set to Single and font size of 10.5, what leading does LO use? 1pt? 1.5pt? 2pt?  With that info, I can know  how much vertical space I need for the no. of lines I want. (Because I have inter-paragraph spacing set to 0.)

I just spent 15 mins googling and searching in the LibreOffice help wiki unsuccessfully trying to find that information.

My best guess is that the paper height will be the Page height.
That the Top Margin and Bottom margin specify the vertical space allocated for those parts of the page, and that the Footer and Header are irrelevant, because they specify the layout of Header (Footer) elements *within* the Top Margin (Bottom Margin).  (I assume that in Heading, Spacing gives the distance from the top edge of the paper to the baseline of the header elements. Hmm, but then I can't guess what the Height is used for. Whether the header elements are measured from the edge of the paper, or from the edge of the margin, I don't understand why you need two numbers, so I clearly don't understand that.

The Help page for Header: https://help.libreoffice.org/Common/Header is hard to follow.  A diagram would really help!  But I think it 
means the Convert 8 page style is a bit weird: Height is the total height of the header (2.9pt only in Convert 8!) but Use Dynamic Spacing allows that distance to be exceeded.

It says the Height is the height of the header.  Spacing (23.1 pt in Convert 8 page style) is the distance between the bottom edge of the header and the top edge of the document text. So it's a relative number: sounds like it adds therefore to the Height.

This doesn't seem to match what I see, so clearly I'm not understanding it, and I'll need to go and do some experiments.  I can't work it out from the Help page.  In my doc, it looks more like spacing is the gap between the edge of the paper and the baseline of the header contents.  But maybe those dimensions are coming from the contents because Autofit Height is on, and perhaps the info is coming from the Header paragraph style, or possibly from that and the Header Right/Left paragraph styles (Font and Indents/Spacing).. nope, that doesn't match what's in my doc.  I seem to remember there's a Character Style defined for Headings that might override that, too. Or perhaps it's overridden just by editing the text for the header directly...

I'm getting very confused.  I need to go and do some experiments.  I'm still very concerned though that the two documents have different number of lines per page despite having the same dimensions set for all page elements.

Time for experimentation.  On a copy of the doc, obviously.

Thanks at least for pointing me at something that will let me try out some things, and making me realise I don't understand at all how the page layout works.

I changed my Writer to Points to see your settings.

You have your body text Line Spacing set to Single.
Which basically means in effect your leading is 0.
So your line heights are going to be your font character height.

Your current page style settings:
575pt page height
- 29pt top margin
- 22 bottom margin
= 524pt available vertical space

524pt/12 = 43.67 picas (odd number)

524pt / 10.5pt text = 49.9047619047619 lines of text per page (odd number)

525pt / 10.5pt = 50 lines (even number) 

Not sure why you have a 575pt page height
8 inches = 576pt
This is why I saw 7.99in when I still had Writer set to inches.

And this is why when I set the page height to 8in (576pt)
I ended up with 525pts of vertical space 
which fit exactly 50 lines of 10.5pt body text.

You definitely have some manual formatting.
Clear it all out.
This is the only reason some of your pages went to the bottom.
Do Ctrl+A and Ctrl+M ... and look at the doc, your "good" pages are gone.

Think about a vertical grid which is evenly divisible, say 12pts.
Take your 10.5pt body text, add Leading 1.5pt = 12pts.
So any available page height which is full picas (12pts ea.) will fit exactly.
48pica page height (8in) - 4pica top and bottom margins = 40 picas vertical = 40 lines of text exactly.

Your title pages are a little more complicated.
But again if you think about the vertical grid it can all fit exactly.
Your font point size, leading, before/after spacing, etc. should all add up to fit in the 12pts grid.
Your Chapter Title style currently has 14pt font size + 9pt after paragraph = 23pt vertical space.
23 is a prime number which is not going fit in any grid (except 23).
Change your Chapter Title style to 14pt font size + 10pt after paragraph = 24pt vertical space.
That will take exactly two lines in our 12pt vertical grid.

Or you could do an 11pt grid.
Just make sure the title text, body text, and the vertical page size all evenly divide into it.
Actually 10pt body type with 1pt leading may be easier to read.

Or you could just fix the page size (576) which will make the vertical space 525pt
and then change the title styler to fit in 2 lines (10.5pt * 2 = 21pt).
So 14pt font + 7pt after = 21pt.

"Normal" would be 10/12 - 10pt font size + 2 pt leading = 12pt line height
Your text is currently kinda cramped - makes it harder to read.

Or do whatever. 
Go wild.

The .5 on the body font size can make the calculations interesting...be aware.


I would get rid of all those page styles - you probably only really need three.
And rename them something descriptive rather than "Convert 1, 2, 3, 4, 5, 6, 7, 8, etc."


So ....
 - set your page size properly
 - set your margins to give yourself an evenly divisible vertical space
 - change all your text styles to fit in your vertical grid

Your pages will fill exactly every time.

You really need fix these page styles.

I was looking at page style Convert 8 because that was the one I saw when looking at page 192 (one of the good pages).

It appears different chapters use different page styles.

Chapter 12 uses Convert 8
- Top margin: 29pt
- Bottom margin: 25pt

Chapter 13 uses Convert 2
- Top margin: 25pt
- Bottom margin: 25pt


So I checked the applied page styles

Convert 1
- Top margin: 25pt
- Bottom margin: 25pt

Convert 2
- Top margin: 25pt
- Bottom margin: 25pt

Convert 3
- Top margin: 25pt
- Bottom margin: 25pt

Convert 4
- Top margin: 25pt
- Bottom margin: 25pt

Convert 6
- Top margin: 12pt
- Bottom margin: 12pt

Convert 7
- Top margin: 25pt
- Bottom margin: 22pt

Convert 8
- Top margin: 29pt
- Bottom margin: 25pt

Default
- Top margin: 25pt
- Bottom margin: 25pt

(So all the rest of those page styles Convert 9 to Convert 18 are not even being used and can be deleted.)

It appears that most chapters are Convert 8.
But Chapters 10 and 13 are Convert 2 (with different margins).

Wadda mess.
You need to organize your page styles.

All that information is super, super helpful.

I will get started on fixing it all up according to your information.

Do you have any idea, though, why it all worked fine in the older version of the document, even though all numbers in the page style dialogs are the same?  It seems strange to me that it suddenly and so dramatically went from "all good" to "99% bad".

What units does LO use for storing these numbers?  I ask because I wonder if the numbers might only *look* the same because they're being rounded before being displayed as points (so maybe, if I selected some other units, I would see differences between the two documents)?  In which case, when I used Reset on some Page Style tab, perhaps I "invisibly" changed a number?

Also, do you have any tips for reassigning italics to each span of text throughout the entire book, after clearing all manual formatting?  Doing so manually will be about a full day's work, and will be quite error prone, too.  I would guess there are something like 2,000 instances to re-apply when I wipe out all formatting.

I assume I should not simply copy and paste all the text into a new empty document with well-designed page styles?  Nor save as RTF or .docx or something first, and then copy/paste from there into the new and pristine document?

Is there any way to apply italics to text that would persist through a "clear manual formatting"?  I have a vague feeling I've seen something about manually setting a word or phrase in italics is not preferred (and, maybe, that defining a character style would be preferable), but I don't know of a simple and efficient way to set emphasis except Ctrl-I or clicking the Italics icon.  I can't believe I'd need to open the Character style inspector and manually select a defined "emphasis/italic" style for each selected piece of text, as that sounds like a many-click sequence of actions.

Anyway, the information you've already provided should be enough for me to fix up my problem, so thank you for all your help already!

These extra questions are just to try to work out how to reduce my work in fixing my problem.

I have no idea why it has changed the pagination, etc. - there is sooooo much stuff going on here that it is impossible to point at one specific issue.
Your original premise that "all page style settings are identical" is not true as described above.

Regarding how LO is storing the numbers, I do not know.
As I mentioned above I did see an issue with fonts. 
See: 
Bug 83090 - UI: Entered font sizes (e.g., 12.4 pt, 12.7 pt) are being rounded off
I have not tested (yet) to see if that affects any other settings.

Regarding changing manual formatting to a character style, no easy way to do this (in LibreOffice).
I posted yesterday regarding this very issue. 
See:
Bug 106556 - Add functionality that highlights all directly formatted text


I did do some additional examination and testing with your document.
There are many fonts (e.g. Courier) in the content.xml file which are not in the styles.
This tells me they are floating around in some manual formatting.
Removing all manual formatting cut the content.xml file from over 12,000 lines to under 7,000 - almost half.
You may want to bite the bullet and do this.
Then properly add a character style(s).
You can assign the style(s) to shortcut keys to make it go faster.

I did play with the page style.
The majority (99%) of your pages are the same page format.
So I set the Default page style properly, and then deleted all the ConvertX page styles.
When you delete a style it goes back to the default.
Then I could play with the one page style and watch what happened.

Get that page style working properly, then add a couple page styles for other things.
 - Title page
 - Front matter - for copyright, etc.
 - First - for first page of the chapter if needed
That's all.
You certainly do not need Convert1-18.

Chapter Title
 - the chapter style has no effect ... because
 - there is no actual chapter title text
 - and the font on chapter prefix is set to Header Char
 - chapter title style that is centered should not have indents set
 - do not use blank lines to position your chapter titles on the page
 - set a before paragraph spacing or a page style (precisely in pts)

NEVER use blank lines for formatting, ever.

Get rid of all the 2-spaces between sentences. Old antiquated typewriter habit.
Find/Replace, or use the Pepito Cleaner extension.
Pepito Cleaner will also find and delete blank paragraphs.

Look at all the applied styles (the selector at the bottom).
You have styles applied which I do not think should be there.
You can use the AltSearch extension to find/replace styles if needed.
Get rid of custom styles you are not using (select Custom at the bottom).
Just delete them and any text will return to the default style.


You could try a creating a new document. 
You are really only using a few styles.
Won't take long.
Then copy everything (Ctrl+A) and Paste Unformatted Text
99% of the text is the same so it could be Default or Body Text.
Ctrl+A then click the paragraph style.
Apply style to the chapter titles.
Applying the character style would be the big time consumer.

Or just clean-up the page styles and paragraph styles and see how it looks.

Good luck.

I am going to change this "Bug" to resolved.

With the exception of your note about the two spaces (an odd new myth, see https://toeinthebookocean.blogspot.com.au/2015/08/punctuation-one-space-or-two-sir.html or http://www.heracliteanriver.com/?p=324), I can see the sense in everything you say, and once again your advice is and will be enormously helpful.

For the time-consuming step of replacing all the italics, I'm going to try to do a find+replace to insert a marker, e.g. {{{ at the start of a span of italics and a }}} at the end of the span, and then directly edit the contents.xml afterwards to replace that with the correct markup for italics.  I will have to directly edit the contents.xml to insert them.  From memory, LO doesn't provide a way to reliably find a character style. (Yes: I just re-tested: it's badly broken: it doesn't find some italics, and it identifies some spans of regular text as italic.)

I just want to say thank you again for looking further into this: I will definitely do as you suggest.  You've been enormously helpful.

Yes, I'm happy for you to close this as Resolved.

NOTE

One last thing that may be of interest: because the one thing that still puzzled me was why the old file was okay and the new one not, this morning it occurred to me to take a copy of the old file, delete all the chapters, and paste back in all the chapters from the new version.

Interestingly, it basically fixed the problem: with a few exceptions (apart from the pages with a chapter heading), 99% of the pages are formatted as they used to be, no longer leaving a blank line at the bottom of the page.

But I still plan to go ahead and follow all your suggestions to produce a clean document with sensible page styles, and just those I need, well-named.  Continuing to work from a mixed-up document that may "explode" on me at any time would be foolish!

Thanks for helping Luke out, LibreTraining guy :) Tweaking status to WFM