Scribus Forums

Scribus => Scripts and Plugins => Topic started by: a.l.e on January 22, 2023, 05:09:20 PM

Title: a python script for the table of contents
Post by: a.l.e on January 22, 2023, 05:09:20 PM
for years i have been telling people not to use the table of contents tool provided by scribus.

i even worked on a new table of contents tool, which was based on the paragraph styles used in the document.
(the proof of concept did work, but among other factors, the scribus team was not willing to replace the current tool with the new one i was proposing... and i was not so eager to make the UI even more complex with two different table of contents tools!)

two weeks ago, pmj proposed a patch to support multiple levels of headings in the current TOC tool:

https://bugs.scribus.net/view.php?id=16887

i suddenly thought that it might be possible to create a script that would fill the attributes by looking at the paragraph styles used  in the document. and make the TOC tool a little less painful.
i wrote the script and it did work well, with a better patch for multilevel TOC it would have made the TOC tool usable... but why should i have stopped there?

here is a python script that runs inside of scribus and collects the paragraphs with specific styles (h1, h2, h3... but you can easily modify the global variable to support any naming schema you see fit) and creates a multilevel table of contents in the current frame:

https://github.com/aoloe/scribus-script-repository/tree/master/table-of-contents

it should work with any current version of scribus (except 1.4.x of course! that's not current!).

i still really think that the current TOC tool should be removed from scribus and replaced by one that is based on styles, but at least, now, there is a solution for creating a TOC in Scribus!

it does not cover all possible table of contents, but i hope that it can allow people to create (a bit more than) basic table of contents with scribus!
Title: Re: a python script for the table of contents
Post by: PatJr on January 22, 2023, 06:02:40 PM
Thanks for your hard work.
Title: Re: a python script for the table of contents
Post by: utnik on January 22, 2023, 07:49:23 PM
hi ale

i downloaded your script. but a test with scribus 1.5.8 and 1.7.0 reverted this:

Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "<string>", line 60
    match section['format']:
          ^
SyntaxError: invalid syntax


utnik
Title: Re: a python script for the table of contents
Post by: a.l.e on January 23, 2023, 10:38:50 AM
hi utnik

ok, yes, that is what i feared.
that lines needs a current python version and scribus seems to distribute a somehow older one.
(it works on debian testing with the system python...)

i will modify those lines tonight and use a more conservative syntax.

thanks for testing!
Title: Re: a python script for the table of contents
Post by: a.l.e on January 23, 2023, 07:49:18 PM
i have now removed the match / case that is invalid in the older versions of python.

... and fixed the README files (during the duplication of the table-of-contents script some of the edits went to the wrong folder...)

let's hope that the script now also works with the python version distributed with scribus : - )
Title: Re: a python script for the table of contents
Post by: utnik on January 24, 2023, 12:39:31 PM
hi ale

i checked your new version with scribus 1.5.8 and it works flawlessly as long as the title is the first paragraph in a frame and there's only one title per frame and you don't use linked frames. (...or unlink them before the title...)

as i often work with the following syntax:

    'h1'-title
        'h2'-title
            text
        'h2'-title
            text

i checked this as well. it needs some manual work (deleting unneeded stuff) – but in general it creates a clean toc.

utnik
Title: Re: a python script for the table of contents
Post by: a.l.e on January 24, 2023, 02:51:42 PM
hi utnik

thanks for testing

can you please provide a sample .sla file?

here it also detects multiple headings per frame

h1
p
h2
p

and i thought that it would work with linked frames, but -- indeed -- i have not tested it.

ciao
a.l.e

p.s.:

h1
h2
p
h2
p

also works
Title: Re: a python script for the table of contents
Post by: a.l.e on January 24, 2023, 06:07:52 PM
indeed, linked frame were not detected (correctly).

i've now pushed a new version with support for linked frame : - )
Title: Re: a python script for the table of contents
Post by: a.l.e on January 25, 2023, 10:18:14 PM
some more work...

- scribus is returning the frames in their order of creation... now, i'm sorting them by position before looking for the heading styles.
- if the heading and paragraph styles do not exist, they are automatically created
- if you want to use specific styles or create multiple table of contents with different styles, now you can define the styles to be used for a specific TOC, by setting the attributes of the frame (see the README file for some short instructions)

also, craig has added an API function for getting the "real" page number (formatted according to the settings in the section). if it works correctly and it has also been added to 1.6, then i will use it also for the TOC script (and remove the code faking the sections as soon as most people can have a scribus version with that function)

the current version of the script is (still) here:

https://github.com/aoloe/scribus-script-repository/tree/master/table-of-contents
Title: Re: a python script for the table of contents
Post by: utnik on January 27, 2023, 12:49:03 PM
hi ale

i tested the latest version of your script vith scribus 1.5.8 on macos 11.7.2. (see attached file)
there are two strange things:
it's no big deal to maintain manually but there might be an easy fix...

thanks for your work
utnik
Title: Re: a python script for the table of contents
Post by: a.l.e on January 27, 2023, 01:51:31 PM
hi utnik

i've patched the script to avoid the empty line at the end.
thanks for pointing to this "small" glitch : - )

on the other side, in my document, i seem to get the toc# styles to be correctly applied in the toc.
i'm using a somehow actual version of 1.6svn but it should not make a difference...

so, now, i wonder what is in the file you planned to attach : - )

ciao
a.l.e
Title: Re: a python script for the table of contents
Post by: utnik on January 27, 2023, 04:54:53 PM
oh - sorry!

here is the attachment...
Title: Re: a python script for the table of contents
Post by: a.l.e on January 27, 2023, 10:50:26 PM
it worked here also with your document.

both with 1.6 and 1.5.8...

mmm....
Title: Re: a python script for the table of contents
Post by: utnik on January 28, 2023, 01:01:03 AM
hi ale

there is something really strange: with scribus 1.5.8 i have to apply 'toc2' and 'toc3' manually after the toc is created. with scribus 1.7.0.svn (r25199) it looks the same in the first place. (the whole toc in 'toc1') but when i press 'cmd' + 'z' the styles are there as expected...

utnik

edit: after pressing 'cmd' + 'z' three times most of the toc reformats in 1.5.8 as well. but the last two entries in my test file switched to 'default paragraph style'.
i will test on an other mac to see if it is something unique on my machine...

utnik
Title: Re: a python script for the table of contents
Post by: a.l.e on January 28, 2023, 10:11:30 AM
(https://forums.scribus.net/index.php?action=dlattach;topic=4480.0;attach=4315;image)

i wonder if the script is 100% correct.

how do you get it?

here is a way that should give you an exact copy of what is on github:

(https://forums.scribus.net/index.php?action=dlattach;topic=4480.0;attach=4317;image)
Title: Re: a python script for the table of contents
Post by: utnik on January 28, 2023, 05:52:06 PM
hi ale

something was broken on another level.
after a restart of the computer your script works as it should.
i'm sorry!

thank you for your work!

utnik
Title: Re: a python script for the table of contents
Post by: sersha on April 25, 2023, 01:08:31 PM
First, let me thank you for the very much needed element in Scribus namely TOC creation! I am producing a book over 300 pages in Hindi (and I do not even speak the language) and it went rather smoothly except for footnotes, most of which I set by hand.
Now I came to the TOC, suffered a bit with the built-in function, then, rather frustrated, came upon your script! I downloaded it per your instructions, then I created styles h1->h5, then toc1-toc5, edited the script to include the headers, marked the headings accordingly in the text, then run the script after the selection of a placeholder linked text frames... but no luck. Out came the following:

Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "<string>", line 163, in <module>
  File "<string>", line 139, in main
  File "<string>", line 82, in get_frame_headings_by_style
scribus.WrongFrameTypeError: Cannot get text of non-text frame.

------------------
Why could that be, and what could have been done to make it run?
Thank you again very much for the valuable work! Best regards, Sergey
Title: Re: a python script for the table of contents
Post by: a.l.e on April 25, 2023, 01:34:01 PM
i did not test the script with the toc in linked frames.
i did it now and it does work.
that's not the issue

the error says that the script is trying to do "selectFrameText()" on a frame that is not a text frame.

the script is not thoroughly tested (but thanks to utnik at least tested) and there might be some edge cases that are not correctly handled.

if you share the .sla file with me (us) i can try to find which frame is the culprit and why...
or you can try to debug it (and share the results so that i can improve the script) by printing the items while the script goes through and checking what is special in the frame that fails.
Title: Re: a python script for the table of contents
Post by: sersha on April 26, 2023, 07:16:37 AM
Thank you so much for your prompt reply!
I enclose the exported working file minus pictures, together with fonts used. In the beginning of the document, I left some linked text frames for TOC. I also enclose my amendments to the script, just in case I spoiled anything inadvertently. Best regards, Sergeytable-of-contents.py.zipEkachakra no pictures.slafonts.zip   
Title: Re: a python script for the table of contents
Post by: a.l.e on April 26, 2023, 09:23:28 PM
no idea why i got to the idea that item[1] == 5 is for linked text frames.

anyway, simply using item[1] == 4 only works well:

https://github.com/aoloe/scribus-script-repository/commit/03c96173d8142c05437c11f09bf2ea759751de75

the way i debugged this, was by adding

print(scribus.getSelectedObject())
at the beginning of get_frame_headings_by_style() ... and, after starting from a terminal, it told me that the last item it saw, was a line...

et voilà!

you can make the same change to your version of the script or download my script again and adapt it.
Title: Re: a python script for the table of contents
Post by: sersha on April 27, 2023, 02:15:14 PM
Great, thanks! Already it worked wonders and saved my day! Some small questions remain:
I am positively amazed how nicely the script positioned all the 5 levels of TOC (see the ch 4 as example). And thank you for hinting how to print out the found header frames, I thought to ask but hesitated. Anyway, I am totally new to Python... Appreciate you work a lot! Sergey.
Title: Re: a python script for the table of contents
Post by: sersha on April 27, 2023, 03:57:36 PM
I do not know how, but I eliminated the "smearing" of the bold. It was a remnant of a previous style for headers left between lines.
So, few, maybe obvious rules.
One header, one line of a particular style. Otherwise and quite logically, it is considered to be yet another entry.
Eliminate unnecessary styles left over from previous experiments. They can play a joke on you!
I still cannot figure out the cause of the page break after the Ch 5...
But the rest seems to be coming out just fine! Enclose the latest results so far.
Regards, Sergeyno pictures p1-5.pdf
Title: Re: a python script for the table of contents
Post by: sersha on April 27, 2023, 05:28:31 PM
At last, a success! A near perfect TOC. The page break after the chapter 5 disappeared when I unlinked frames between second and third headers after the chapter title, respective pages 237 and 238. I do not understand, why it worked this way, but it did.
Now, the only thing left for the perfection would be the transfer of appropriate Roman numeration... I enclose the last sla and TOC.
Thank you again, a.l.e!
Title: Re: a python script for the table of contents
Post by: a.l.e on April 27, 2023, 09:42:12 PM
voilà, i've pushed a much shorter version of the script that uses the page numbers as defined in the document sections (not in the sections variable).

https://github.com/aoloe/scribus-script-repository/tree/master/table-of-contents

the script checks if the new function is available and then uses it.

if it's not available, it will show page numbers from 1 to n.

(you will need a scribus newer than the 24. of january 2023 to get the page numbers as defined in the sections)

if you want to use the section structure with an older version of scribus, you can use this "older" version of the script (the last one with the "sections" variable:

https://github.com/aoloe/scribus-script-repository/tree/03c96173d8142c05437c11f09bf2ea759751de75/table-of-contents