Scribus Forums

Scribus => Text and Typography => Topic started by: mujeebcpy on July 10, 2019, 01:18:34 PM

Title: Scaling width of charecters in unicode (Malayalam, India)
Post by: mujeebcpy on July 10, 2019, 01:18:34 PM
we are implementing scribus for a news paper organization in Kerala, India. after 1.5 version Unicode Malayalam is rendering correctly. hyphenation is also working well. after lot of trail lay-outing we facing a problem in scaling width of characters. when Unicode fonts scaled all proportion of the fonts get distorted.  i tried the same font in indesign which works very well. text condensation is a much needed feature for a news paper. it will be very helpful if that issue fixed. im including sample unicode malayalam word with this. try scaling width.

ജോലിയുള്ളവരുടെ കോഴികള്‍

we are using scribus 1.5.5 latest build until today

[attachment deleted by admin]
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 10, 2019, 03:28:56 PM
hi mujeebcpy,

nice to see newspapers considering scribus!
and even nicer knowing that anil from atps.in had a try several years ago to produce a version of scribus that would support malayalam but "failed" to make the solution solid enough for being included in the mainstream scribus!

i tested your text into a document with Noto Sans Malayalam and i can indeed see the issue.

can you please open a ticket in the bug tracker https://bugs.scribus.net ?

ciao
a.l.e
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: mujeebcpy on July 10, 2019, 03:51:10 PM
Thank for the quick response. i know Mr.Anil. and i will open the bug ASAP. i am including a part of our layout work here (it is just sample contents. so im not including the head portion)
one more thing, the dropcap feature need 3 characters for some letters in Malayalam.


[attachment deleted by admin]
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: ranjithsiji on July 10, 2019, 04:19:36 PM
Quote from: a.l.e on July 10, 2019, 03:28:56 PM
hi mujeebcpy,

nice to see newspapers considering scribus!
and even nicer knowing that anil from atps.in had a try several years ago to produce a version of scribus that would support malayalam but "failed" to make the solution solid enough for being included in the mainstream scribus!

i tested your text into a document with Noto Sans Malayalam and i can indeed see the issue.

can you please open a ticket in the bug tracker https://bugs.scribus.net ?

ciao
a.l.e

Myself Ranjithsiji, Working with Mujeeb. We are trying to implement Scribus (Infact the whole Free Software Stack Scribus, Inkscape, Gimp, Krita) into various Newspapers and magazines in Kerala. We have a small dev team with some experience in Qt programming. If you can help us to fix this bug or help us in a direction like what to do to fix this bug will be verymuch helpfull for us. Expecting help.

A Bug is created at https://bugs.scribus.net/view.php?id=15736
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 10, 2019, 06:15:51 PM
nice to meet you, ranjitshiji!

for this specific issue, we well probably need to wait for jean to come back from a few days holidays.

i'm not a font specialist, but i guess that in this case you will have to focus on the font shaping and -- probably -- harfbuzz.

it might something simple, that jean can fix very fast... but if it's something that needs more work, i would be glad to help you guys better understand this part of the scribus code that is so important for you!

i have also added another screenshot to the ticket, one that better shows the issue for people like me who have no clue about your language : - )
(i was curious about the meaning of the sentence and i was a bit surprised about what google translate told me...)
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: mujeebcpy on July 10, 2019, 08:25:51 PM
//(i was curious about the meaning of the sentence and i was a bit surprised about what google translate told me...)//
haha. i needed the ജോ and കോ letters to show the extreme problem with scaling. so i just choose two words ;) thats all
by the way the bug i reported is not loading now
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 10, 2019, 10:26:55 PM
hi ranjithsiji,

i had a chat with andreas about this bug.

he asked me about the pdf output and i tried it out: when you increase the character width, the second character wanders to the right, in the pdf it's displaced to the left.

according to andreas, the relevant variable is "xadvance"...
i wonder if looking at differences in how pdflib_core.cpp and text/screenpainter.cpp manage it, might help fixing the bug...

he also supposes that the reordering of the characters might be "confusing" scribus...

grepping for xadvance in the source code gives only a few occurrences and might help studying how this work.

i'll keep you informed : - )

ciao
a.l.e
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 10, 2019, 11:04:26 PM
Further feedback: "it's not the reordering, it's the clusters. Scribus doesnt scale multi-glyph clusters correctly"
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: ranjithsiji on July 11, 2019, 03:39:50 AM
Quote from: a.l.e on July 10, 2019, 11:04:26 PM
Further feedback: "it's not the reordering, it's the clusters. Scribus doesnt scale multi-glyph clusters correctly"

Yes. It is the problem of scaling the multi-glyph character. ജോ is a combination of ജ + ോ. When we scale the sentence the calculation of spacing creates the problem.

I tried this thing in Libreoffice and it is working perfect there. Is scribus using same library to display text? Can we get any help from their code?
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 11, 2019, 07:53:48 AM
hi mujeebcpy,

if you look at the patch andreas has put in the bug tracker you will notice that scribus uses cairo for the display on screen and its own library for creating the pdf.

the patch is "very simple" and studying it will help you understand a bit how scribus works in a part of the code that is very important for indic and arabic languages : - )

currently, i have to first get my own copy of the scribus code to compile again, then i can test if the changes proposed by andreas work for me!

i'll get back to you very soon
a.l.e
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: ranjithsiji on July 11, 2019, 08:41:43 AM
hi ale,

I tested the patch that andreas given with latest scribus code and it is working perfectly. I amazed about the solution. This was real life saver for us. Thank you for your quick response and patch. I think this will fix the problem for all indic langauges. Because Malayalam is the most complex indic language with lot of complex rules. So it may work for all.

Thanks for this amazing fix

Best,
Ranjithisiji
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: Nermander on July 11, 2019, 04:44:04 PM
I think this really shows the power of Open Source development.
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: RodneyLee on July 11, 2019, 06:40:10 PM
if its just "Headlines" why not do a .svg in inkscape, scaling would be perfect
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: mujeebcpy on July 11, 2019, 07:31:02 PM
Quote from: a.l.e on July 11, 2019, 07:53:48 AM
hi mujeebcpy,

if you look at the patch andreas has put in the bug tracker you will notice that scribus uses cairo for the display on screen and its own library for creating the pdf.

the patch is "very simple" and studying it will help you understand a bit how scribus works in a part of the code that is very important for indic and arabic languages : - )

currently, i have to first get my own copy of the scribus code to compile again, then i can test if the changes proposed by andreas work for me!

i'll get back to you very soon
a.l.e

yes. i build and run it successfully. truly speaking i didn't expected this much fast response and fix. i totally wondered. and ofcourse  i will try to understand the code. expecting guidance from all of you
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: mujeebcpy on July 11, 2019, 07:38:29 PM
Quote from: RodneyLee on July 11, 2019, 06:40:10 PM
if its just "Headlines" why not do a .svg in inkscape, scaling would be perfect

we suggested that. and the layout workers are little bit worried about the total shift from pagemaker. this migration plan is going on about 6 months. main problem was condensation. in news paper lay-outing last column shoud be aligned. to make this some condensation is needed in last paragraph of body text too.
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 11, 2019, 09:27:42 PM
so, as jean and andreas discussed in the bug tracker, the patch does solve a short term problem but is not a complete solution.

if you read a bit their comments in the bug tracker and try to read the code around the patch already proposed by andreas, you will already have an idea about how it works.

don't hesitate to ask for details... i might be able to help or can ask further...
(or somebody from the team might show up in here... or you might join also the irc channel for more interactive discussions)

just a side note: as far i can tell, for latin fonts it's very unusual to create condensed "text" by changing the width of the letters... you normally use a condensed variation of the font... and if you just try condense a font by 20%, you don't get a good result. of course, i cannot judge how good the result is for the noto / malayalm font.
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: avox on July 11, 2019, 10:13:54 PM
Quote from: mujeebcpy on July 11, 2019, 07:31:02 PM
...
yes. i build and run it successfully. truly speaking i didn't expected this much fast response and fix. i totally wondered. and ofcourse  i will try to understand the code. expecting guidance from all of you



Quote from: ranjithsiji on July 11, 2019, 08:41:43 AM
hi ale,

I tested the patch that andreas given with latest scribus code and it is working perfectly. I amazed about the solution. This was real life saver for us. Thank you for your quick response and patch. I think this will fix the problem for all indic langauges. Because Malayalam is the most complex indic language with lot of complex rules. So it may work for all.

Thanks for this amazing fix

Best,
Ranjithisiji

You are welcome. I'll have a closer look over the weekend re. PostScript output and other cases.

I usually don't hang around here, so if you want a response from me, you either have to alert a.l.e or send a mail to scribus-dev :-)
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 15, 2019, 08:32:23 AM
the patch by andreas and jean is now in the current scribus code (trunk / master).
if you compile scribus, you can get it trough svn and git.

an appimage can be found (during the next week) here:
https://gitlab.com/scribus/scribus/-/jobs/251287685/artifacts/browse

the PPA provided by ubuntu should also already contain the fix.

and jean provided a snapshot for windows: https://sourceforge.net/projects/scribus/files/scribus-svn/1.5.5.svn/
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: mujeebcpy on July 15, 2019, 10:42:34 AM
Hi, i pulled and built. now everything works fine. svg out also rendering correctly. thanks for the help and fix.
i have another bug to report with Dropcap. should i start a new bug?
Title: Re: Scaling width of charecters in unicode (Malayalam, India)
Post by: a.l.e on July 15, 2019, 05:50:39 PM
yep, please open a new ticket for it...

and if you think that your team has the skills to fix such bugs (now that you have seen how the other bug has been fixed...), don't hesitate to try to dive into the scribus code!