Scribus Forums

Scribus => General Discussion => Topic started by: PK on January 23, 2024, 10:06:34 AM

Title: [TOC] get an attribute directly from a style
Post by: PK on January 23, 2024, 10:06:34 AM
Hi,

For doing a TOC, you need attributes as entry points for the table. This is a heavy and repetitively job to do by hands, especially when you have a regular structure like a book, which is governed by fixed rules like styles.

Is it possible to (easily) associate an attribute with a dedicated style, with the content of the style (i.e. all the characters that match the style) as the value?

I think this is not too much difficult to develop as a script, but I can't imagine that is not already done  ;D
Title: Re: [TOC] get an attribute directly from a style
Post by: PK on January 23, 2024, 11:58:13 AM
It seems already to be done:

https://github.com/aoloe/scribus-plugin-tableofcontents/tree/master
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on January 23, 2024, 12:12:06 PM
nö, that won't do it.

this one will do it:

https://github.com/aoloe/scribus-script-repository/tree/master/table-of-contents

there is a forum thread about it:

https://forums.scribus.net/index.php/topic,4480.0.html

in the long term, i'd like to have something built into scribus, but i'm not sure i want to do that as long as there is no commitment on replacing the current one with that.
Title: Re: [TOC] get an attribute directly from a style
Post by: PK on January 24, 2024, 08:00:51 AM
Giusto  :)

Your script is nice and works perfectly.

I notice a light bug, but may be it comes from me and the way I run Scribus.

When I need to force a section not to stay on the bottom of a page, for not having a orphaned title, I add an frame jump before, to move it at the top of the next page. Your script considers this sign like a part of a title (its style is the same as the title, because I place it on the same line, in front of the value of the title) and adds an entry in the TOC (so, a wide entry!). There is also a Scribus bug: when the TOC entry value is wide, the next entries are not printed...

May be must I correct the way I run Scribus in inserting an other jump sign?

Thanks in any case: very good job  :D   
Title: Re: [TOC] get an attribute directly from a style
Post by: PK on January 24, 2024, 08:07:04 AM
Correction: this is not a wide entry, but an entry with the frame jump, so the TOC is divided as many times that there are frame jump entries. So, there is no bug for Scribus in the TOC: all the entries are well here, but there are in many text frames...
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on January 24, 2024, 07:59:35 PM
i have updated the script to ignore the column and frame breaks:

https://github.com/aoloe/scribus-script-repository/tree/master/table_of_contents

(and i've also renamed the script to match the python conventions...)

can you please test it and, if you find other bugs, upload a sample .sla file with the specific text?
Title: Re: [TOC] get an attribute directly from a style
Post by: PK on January 25, 2024, 06:45:15 AM
It is OK like this: nice work  8)

You changed the name of your script by transforming dash in underscore, is it normal?

You may add an other functionality. I have added in the titles sometimes line breaks for a perfect layout when the title are too long. I removed it from the toc, because the styles are very different and the police is smaller. I don't know if it is a generic functionality indeed!

Thank you.

I don't understand why your scripts are not delivered with Scribus...
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on January 25, 2024, 08:25:34 AM
concerning the move to _: i'm renaming all my scripts to use _ instead of -... personally, i prefer the dashes, but python advocates the use of _ (and the linter keeps telling me that).

when developing the integration into the menus (https://bugs.scribus.net/view.php?id=15579) i decided to only actively support what PEP says and to slowly rename all my scripts to match that.

concerning the new lines: i have not worked with table of contents in many years but, yes, i agree that it should at least be an option to remove them from the titles.
i just wonder a bit how to do it.

should be the default or if it should even be the only behavior?

a possible behavior might also be, to drop what is after the new line (i think i have had such cases in my distant past...)

and, finally, about the scripts distributed with scribus: personally, i don't think i've ever used any of the scripts that are coming with scribus.
i'm not fond of scripts that have complex UIs and try to cover every possible workflow. i prefer scripts tailored to specific tasks.

imo, scribus should make it easy for the user to run

- their own scripts
- scripts from collections they like
- scripts provided by their "IT".

my patch adding the user's scripts to the navigation goes in this direction... but who knows when well it will be reviewed...
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on January 25, 2024, 11:37:20 AM
the script now optionally strips newlines or truncates after the first newline...

one can pick the option by setting the attribute on the target frame.
Title: Re: [TOC] get an attribute directly from a style
Post by: PK on January 27, 2024, 03:05:49 PM
OK. Works perfectly.

Nice job: thank you!
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 18, 2024, 10:55:27 PM
1.7.0.svn now has a style based TOC generator that I added this evening.
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 19, 2024, 04:55:01 PM
hi craig

nice start.

personally, i still believe, that the attribute TOC must go away. it adds useless complexity both in the code and in the UI.

what are your plans for further features?
the current state does not seem production ready yet : - )

toc-mrb-01.png

p.s.: you can read this thread and https://forums.scribus.net/index.php/topic,4480.0.html to find some edge cases that need to be supported... (the README and the python script might also be helpful...)

toc-mrb-02.png
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 19, 2024, 06:31:42 PM
Please share those files. It looks like you have a text frame style and a style on text.
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 19, 2024, 06:39:37 PM
I also uploaded a sample file on
https://bugs.scribus.net/view.php?id=5975

Working well there

Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 19, 2024, 06:52:59 PM
(https://bugs.scribus.net/file_download.php?file_id=12445&type=bug)
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 19, 2024, 10:37:59 PM
i can attach my file, but it won't really show you, the few essential things you likely need to take into consideration for a "useful" TOC in scribus.

as i wrote, it would probably be helpful if you could go through:

you will then have a list of things that people actually using scribus need for creating a TOC (their feedback has been very very useful to me!) and have a rather simple reference implementation.
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 21, 2024, 11:02:32 AM
Today, I have a bit more time.
Here my first impressions about the TOC in Scribus:



The Config > Tables of Contents dialog is not comfortable to use and some important features are missing.
(Some of the issue might already have been there before introducing the styles in the TOC...)

The top half of the dialog is by default empty and stays most of the time almost empty:
the huge majority of document with a TOC will only have one single TOC.
And when the user gets for the first time into this dialog, the goal is to define a table of contents!

I'm suggesting to pre-fill the list of TOCs with "Table of Contents" (without the "1"! The second one gets a "2") and make the list much smaller.
I guess that the TOC should only be saved in the .sla, if it has a minimal definition (at least one level defined...).

On the other side, the most "important" part of the dialog is probably the matching of heading and toc paragraph styles.

For each level we need (at least):


So the first suggestion is to make the list of TOCs a simple list with two "icon buttons" (+, trashcan) and let the user change the name of the TOC by double clicking the entry in the list (no input field needed).
This list would be shown in the left part and on the rights side the TOC's settings.
The lower part would be for adding the levels with styles (or alternatively the attributes, if after having implemented a "usable" style based TOC you still think that somebody would want to use that...)

It could look somehow like that:

                                                                                                               
++---------------------------+-+-+-+-----------------------+-+                                                 
||table of contents         || |+| | type: [style |v]      | |                                                 
||                          || +-+ | source: [frame |v]    | |                                                 
||                          || +-+ | [ ] non printing      | |                                                 
||                          || |-| |                       | |                                                 
|+---------------------------+ +-+ +-----------------------+ |                                                 
|+---------------------------------------------------------+ |                                                 
||[h style |v] [toc style |v] [page number |v]             | |                                                 
||                                                         | |                                                 
||                                                         | |                                                 
||                                                         | |                                                 
||                                                         | |                                                 
||                                                         | |                                                 
|+---------------------------------------------------------+ |                                                 
|+-+ +-+                                                     |                                                 
||+| |-|                                                     |                                                 
|+-+ +-+                                                     |                                                 
+------------------------------------------------------------+                                                 
(https://textik.com/#153e79c2edb01a7a)

By the way, as you might have noticed, my TOC Python script does not have a UI.
The main reason for it, is that it's indeed not easy to create a good UI and opted for default values and using the attributes.
But here now we have a UI and, as it is now, it forces lot of unnecessary mouse movements and scrolling that can be avoided.



One further important issue: picking the destination frame!
Do you really expect the user to know the name?
I see two possible solutions (which are not mutually exclusive):




I leave it open, if the TOC definition should stay in the document configuration, or if it should be moved to a separate dialog that can be triggered when a frame (the one containing the table of contents!) is selected.

In my script, I went for defining the TOC as a set of attributes on the target frame, so it seems to be possible to define it outside of the settings and it might be even easier to do so!

It might also be necessary to individually update the TOCs, which might be easier with a separate dialog.



Still the UI: the "Generate Table of Contents" menu entry is disabled by default.
I'm not sure if would be a good idea to start the Document Settings, since the user then would expect to use that menu entry for modifying the settings.

It might be worth exploring the idea to add tooltips to menu entries that are deactivated for non trivial reasons (we do not need to explain why you can't load an image on a text frame...).

This seems to to be possible with Qt 6 (not tested):

https://stackoverflow.com/questions/60484006/how-to-include-tooltips-for-menu-items-in-qt

(This is probably not needed if we get two menu entries, one for defining the current TOC, the other for generating all TOCs.)



On top of that, there are features that have been requested and are implemented in the script:




Finally, a few bugs I stumbled upon:




Lot of text, sorry... Probably, just reading the README, the code and the other thread would have been shorter : - )
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 22, 2024, 10:45:14 PM
A lot of those things were already in the works.
I already committed a fix for the duplicate lines a couple of days ago
I've now added styles per heading.
I've never seen a TOC with different page number placement (ie start or end)
- Removed the [Default...] style
- rearranged doc setup ui
- Added remove line breaks code but not enabled yet.. it means joining separate paragraphs.. still to do.
- Removed the 1 from the first TOC naming
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 23, 2024, 10:18:46 AM
hi craig

hard to say something...

your announcement was "Scribus now has a style based TOC"...
and since scribus already had a TOC that the team seemed to like, seeing a style based TOC that basically replicates the old one (with almost all the drawbacks mirrored) gave me bad vibes.

sorry, if i misunderstood your goals.

it might be useful if, the next time, you could also announce the work you still plan to do.



i made a second quick test this morning and, while i see some improvements (the massive duplication is gone):


no idea which of those points are already clear to you and where the feedback is useful.

i attach another simple test file and a screenshot of the UI (since most people won't be able to try this out...)
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 23, 2024, 10:54:05 PM
hi craig

after this afternoon's changes, the result looks better! thanks.
(sadly, there are also some regressions: i don't have any paragraph styles in the lists anymore...)

there is one thing i noticed: you made the removal of the frame and column breaks depend on the "remove new lines" option.

i really think that they should always be removed. there is no reason for wanting them to be copied over into a table of contents (ok, i just found out that it's slightly better to first add the new line and then the frame break: but you can't count on people to do that!)

in my eyes, removing the line ends is rather for cases situations like:

I.
First chapter

or

I. First chapter with a long title
and manual new line
because it's not so nice to have
full width titles

in the attached .odt file, you can see what i think this the right way to present the page breaks in scribus (well, of course, in scribus it will be a frame break).
i would really be important that in scribus too, for the frame break to also be a line break... but that's a different topic!
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 24, 2024, 11:49:49 PM
Ok a new version is now in SVN.
Reorganised the UI for main TOC
Created settings per style header, removed extra style list. Added single style list selector, and individual settings for remove line breaks, page numbering location
Removed the line break removal from the main TOC
COL/FRAMEBREAKs always removed, line breaks per settings (currently defaulted to off)
There's probably a few UI tweaks still to do.
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 25, 2024, 09:00:42 PM
hey!

it starts to work correctly!

i had to recreate my test file (somehow the old one did not work anymore... but it has been edited with multiple intermediate steps of the TOC... so it was somehow expected to happen).

the order of the titles is now what i would expect, but the one that is on the second page (which is in a chain starting on a first page) is showing up twice.

concerning the UI:

- i would move the options that are specific to the attributes in the lower half of the dialog and show one of both panes, depending on the selection.
  that is, if after having implemented all this, you still think that it's useful to keep around the attributes based TOC...

- when no style is selected in the list on the left, the properties on the right should be disabled.

- since now the order of the style is not relevant anymore, it might be not be strictly necessary to move them up and down.
  but i guess that most people will want to see them in a specific order, so it might be good to keep the feature.

finally, the last issue still open concerns the way to pick the target frame:

- on the one side, the user will not now the name and will have to close and open the dialog again to find it out.

- on the other side, a real document might have dozens of text frames and the user will have to scroll though a big list of items...

... i guess that the simple solution is to add a button "select current frame" next to the list of frames, and make it active if a frame is currently selected.

thanks for the fixes!
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 26, 2024, 10:04:11 PM
See how you go now with r26074.
Just the frame selection idea to solve in the future.
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 28, 2024, 08:01:29 AM
good morning craig,

the UI looks ok now.

the detection of the entries for the TOC is mostly good.
but, after having created a new page about mixing chained titled and free standing ones, i noticed that some titles on it were not collected.

on page three of the attached document, the two h2 in the chained text are missing from the TOC.
i guess, that it's because of the empty line before the titles.

finally, the reason why i've created page three:

if i recall correctly, in my python script, i keep the titles from the same page and chain together.
and i sort them by the order in the chain, not by the order on the page.
(not very hard to do: process all the next items if they are on the same page; ignore frames with precedent frame on the same page)
it's probably not something that happens very often, and the example on page three is a bit contrived, but if i think of a newspaper with four/five columns and multiple articles per page, then it's a layout that is somehow common.
it's true that newspapers don't have a TOC, but if, in the (near) future, we manage to also generate a PDF TOC, then the order is relevant.

up to you to decide if you think that this use case is legit and if it needs to be supported in this first version of the TOC.

thanks for your work!
a.l.e
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 30, 2024, 09:34:51 PM
Thanks Ale. I'll see what I can find.
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on March 31, 2024, 06:42:20 PM
Hi @a.l.e

Please check out r26077. Its all good now with your toc test 2 file.

Craig
Title: Re: [TOC] get an attribute directly from a style
Post by: a.l.e on March 31, 2024, 10:08:54 PM
yes, now the order seems to be good!

for me the question stays open, if it's better or not to mix the titles from a chain with other titles in between, but we can for sure wait and see if somebody brings a real world document where this matters.
Title: Re: [TOC] get an attribute directly from a style
Post by: MrB on April 01, 2024, 10:30:12 AM
A few more commits to fix another issue and code simplification in r26082. Your test still passes.