Scribus Forums

Scribus => Text and Typography => Topic started by: Rob Dixon on May 26, 2016, 05:41:17 PM

Title: .sla file structure
Post by: Rob Dixon on May 26, 2016, 05:41:17 PM
I am very much a Scribus newbie. I have my own database package that can generate XML and I wish to import this into Scribus. I think that I know how to do this (I have managed it once!). What I do not know is what to include in the XML for new line codes, style codes, etc. I could easily modify the database package to include whatever is necessary. I thought that I might find some of the answers on the page https://wiki.scribus.net/canvas/Formatxml as this contains a couple of simple samples. I assumed that the definitions of the styles "header 1", "body" etc. would be in
http://www.scribus.net/formatxml but that does not exist. Can someone please point me to documentation that will tell me how these features are structured in Scribus?

I posted this under Beginner Talk but didn't get any response.

Many thanks


Title: Re: .sla file structure
Post by: a.l.e on May 27, 2016, 11:19:51 AM
creating and maintaining a dtd for the .sla file format would be a nice thing.

until such a documentation exists (in a format that can be use to validate files) i don't really recommend messing with the file format.

an alternative is checking the scribus scripter and see if it can do what you need or even improve to make it do it.

there are a few people who have put -- some time massive -- time into hacking directly data into the .sla files and:

- their work breaks everything time the team changes something in the file format
- most of those efforts are abandoned and unusable for other people (since they're tailored to a specific task)
- the scribus project at a whole, does not profit in any way from those efforts.

of course, you're free to create your own solution. but, personally, i prefer spend my time in tasks that make scribus better (or myself hapy... of course :-)
Title: Re: .sla file structure
Post by: Rob Dixon on May 27, 2016, 06:24:40 PM
Many thanks. I haven't made myself clear. I don't want to change Scribus - I want to include in my XML the necessary code that will instruct Scribus how it is to display/send to a PDF the text.

I found a .sla file which contained, for example 


<ITEXT CAB="2" CCOLOR="Black" CBASE="0" CSTW="-0.1" CSIZE="12" CULP="-0.1" CH=" ${invoice.number}&#x5;" CSHADE2="100" CKERN="0" CSCALE="100" CSHADE="100" COUT="1" CSCALEV="100" CSTP="-0.1" CULW="-0.1" CSTYLE="0" CFONT="Times New Roman Regular" CSHX="5" CSHY="-5" CSTROKE="Black" />
   
Is this typical? It seems extraordinarily wordy (although probably very flexible) but surely this must already be defined and documented.  Is it always necessary to define all these values or do any of them remain valid until changed?

What do CAB, CBASE, etc. mean? What are the valid values? Is ITEXT Scribus specific or simply used by whoever created the file? It seems that CH is a variable but what does &#x5; mean? the length of the invoice number? What controls a new line and what controls the space between the invoice number and the next and previous fields? etc.

Any help gratefully received.

Title: Re: .sla file structure
Post by: a.l.e on May 27, 2016, 10:40:31 PM
you made yourself clear.

and because that line is typical, i prefer not to spend much time trying to understand it...
Title: Re: .sla file structure
Post by: Rob Dixon on May 29, 2016, 08:33:39 PM
Thanks but I am confused. If you know that that line is typical, how did you find this out and how did you find out what it meant? There must be documentation somewhere. I just don't know where. Please can you or anyone else help?

Title: Re: .sla file structure
Post by: Nermander on May 30, 2016, 09:16:41 AM
Quote from: Rob Dixon on May 29, 2016, 08:33:39 PM
Thanks but I am confused. If you know that that line is typical, how did you find this out and how did you find out what it meant? There must be documentation somewhere. I just don't know where. Please can you or anyone else help?

Probably the only documentation today is within the source code.

I have never dug into the format, but I remember that an object can be located anywhere in the SLA, the page where the object is placed is a property of the object and does not depend on the structure of the SLA. This means that to move an object to another page, you just change a property of the object. You do not have to move anything within the SLA to move an object to another page.

This might have been changed the last years, I know there have been projects to improve the structure of the SLA format. But the SLA format is not (yet) intended to be human-readable.
Title: Re: .sla file structure
Post by: Rob Dixon on May 30, 2016, 10:15:52 PM
Nermander,

Many thanks. I know that you think that I am following the wrong route but I still want to give it a try, at least for a small document. I don't quite understand quite how Scribus exists if there is no documentation. I don't know how many programmers were involved at the beginning but they must have had some form of instruction. Do you know if there were specifications before coding began or at least ambitions?

In addition, what is the point of having the ability of importing XML if you cannot do anything with it?

Title: Re: .sla file structure
Post by: Nermander on May 31, 2016, 08:06:03 AM
Scribus is still in development, there are many things on the wishlist.

From what I understand the SLA format was not made to be "structured", it was just convenient to use XML because it is an open format and there were good libraries to read and write XML. But I think the SLA format intially was just a way to "dump" the internal working format om Scribus into a file. That is why it is awful to work with.
Title: Re: .sla file structure
Post by: a.l.e on May 31, 2016, 08:51:04 AM
hi dogil,

scribus has of course documentation for the users.

and there is also some documentation for the scripter.

but the .sla file is indeed mostly undocumented.
there are some efforts being done (also by myself) but as you can see from your own behavior it's very hard to get people to help out and contribute to something that really improves scribus.

btw, you *can* process xml from scribus! just not outside of it, but inside of it.
just write a python script that you run from inside of scribus, use a python library to process xml, manipulate a scribus document, produce a pdf.

sadly, most people prefer to go for a quick and dirty external processing that will be by its own nature short living and too few people help maintaining and extending the scripter.
Title: Re: .sla file structure
Post by: Rob Dixon on May 31, 2016, 10:28:39 AM
Nermander,

Surely someone must have defined the structure of the sla format or was it just one coder working by himself?

Title: Re: .sla file structure
Post by: a.l.e on May 31, 2016, 10:33:16 AM
rob,

yes, it's mostly one coder doing some work and then asking for feedback on the solution he is proposing.
if nobody raises his hand, he goes on and the code gets merged...

scribus has a small team... and, sadly, exactly because of the things you're pointing to, it's hard for new people to get in...
some manages to do so... but it's a steep climb...

ciao
a.l.e
Title: Re: .sla file structure
Post by: Rob Dixon on May 31, 2016, 10:42:35 AM
Hi a.l.e

Many thanks. When I generate XML from my database package, at that stage it has a context so that I know what styles, etc I wish to use and could insert them into the XML, but, if the XML has already been exported, the context will have been lost when it is imported to Scribus so a script would not know what to do.
Title: Re: .sla file structure
Post by: a.l.e on May 31, 2016, 11:21:38 AM
hi rob,

if you miss that context, you're probably doing something wrong (like not storing the context in the xml).
hard to say what is not as optimal as it could be, without knowing more about what you are doing.

you also might be able to directly query the database from the python script, if python has bindings for it...
Title: Re: .sla file structure
Post by: Rob Dixon on May 31, 2016, 03:18:17 PM
Hi a.l.e

I patented my database so can claim that it is unique. It is a connectionist NOSQL multi-dimensional database which stores all meta data and user data in one integrated contextual database. It is not a flat file - all attribute iterations are indexed. Both customer and product may have an attribute "name". The database stores separate contextual definitions for the two attributes called "name".  In XML, you could have <customer><name>smith</name></customer><product><name> etc. and I guess that a script could be written to handle this if, say, customer name was to be in bold, etc. But there are an unlimited number of possibilities and a script to handle these could not be universal, and, as far as it went, it would be duplicating the information already in the database. The database is universal and new entity types, attribute definitions and application definitions can be created without any coding as the definitions are stored in the database, rather than in program code. The database is designed for massive data volumes and enormous numbers of concurrent users, and the almost zero second response time does not noticeably change as data volumes increase.

I need to find a way, if possible, of putting style values into the XML - I cannot imagine any other solution, other than importing the XML into Scribus and then doing all the work on styles etc. manually, within Scribus, but this would be very time consuming and a waste of time.

Does anyone have any ideas please? It seems to me that sla is so basic to Scribus that it is hard to believe that there is no documentation of its structure.
Title: Re: .sla file structure
Post by: a.l.e on May 31, 2016, 05:57:05 PM
sorry, you created and patented your unique database and now are you asking -- well almost requiring -- me / us to provide you help for free?

... you created the most astonishing and powerful database and you're not able to create a simple python interface for it? and you're relying on random users on the internet to solve your problems?

please hire a consultant who has a clue about how to process xml data to produce formatted documents out of it.

if you have to process a huge amount of data produced by an enormous number of concurrent users, the least of your problems is to grok how to get a few fields into the scribus .sla.

how do you think you're going to get scribus to produce pdfs out of that massive amount of data? mmmm...
i'm really not sure that scribus is the right tool for the job...
while it can do some automatic processing of data, it's a tool that has been created for interactive usage!

really, if you think that scribus is the tool you need (and in most other cases too, if you want your xml to be processed by external libraries / applications) you should look into your database and check how to tweak it to put in the xml all the information needed to further processing the data.

i don't know, as long as you don't tell us what you're up to, it's really hard to help you.

that was my last reply on this topic, except if you give very good reasons (for me!) to engage further in an issue that i see as hopeless.
Title: Re: .sla file structure
Post by: Rob Dixon on May 31, 2016, 06:57:06 PM
Hi a.l.e

I did not say that I had made any money out of my database. Whilst it can process a huge amount of data produced by an enormous number of concurrent users, it only has a few million records to date. Its 224 bit address space is somewhat under utilised at present.

I should perhaps explain that I only want to create a pdf for a limited part of the database, for instance, for a printed catalogue. A user can make an interactive request for the data he wants to "export" from the database and can select at runtime whether this is to be printed on a PCL 5 local printer or output as XML. He or she can select certain options such as paper size, orientation, prime font size, duplex printing etc. for the printed output and others for XML. The printed output is typeset and of reasonable quality but not adequate for a polished professional looking catalogue.

We all have our skills and our limitations. One of mine is a total lack of knowledge or experience of python or scripting. But that doesn't seem to me to be my problem.  All I want to know is what style codes I need to include in the XML. I don't want to create a python script and if I could find the documentation for CAB="2" CCOLOR="Black" CBASE="0" CSTW="-0.1" CSIZE="12" CULP="-0.1" etc. I could include the necessary code in the XML without much difficulty.

I think that I understand all I need to know about outputting XML for my purposes. It is hardly very complex. I don't know much about processing XML.

Perhaps Scribus is not the right solution, but Adobe InDesign is expensive and I ideally want a generic solution for all applications that doesn't require users to purchase additional software.

If you don't wish to respond again, I understand. I appreciate your input to date. Where else can I go for help?




Title: Re: .sla file structure
Post by: Nermander on May 31, 2016, 07:51:07 PM
It's most likely only explained in the Scribus sources, because it is only designed to be read by Scribus.

But I wouldn't expect to be able to take a structured XML file and apply style codes to get something out of Scribus. You could have to create (and place) a text frame for each page. This means you must calculate how much space your content will take.

I assume you have already found this page?
https://wiki.scribus.net/canvas/File_Format_Specification_for_Scribus_1.4
Title: Re: .sla file structure
Post by: Rob Dixon on May 31, 2016, 09:28:17 PM
Thanks Nermander.

On the page https://wiki.scribus.net/canvas/Formatxml
it says
"The dimensions of the text frame are based on the live area of the document. That is the size of the page minus the margins.
New pages with connected text frames will be created while the current text frame overflows. "

I read from this that I won't have to fit my XML to each frame separately. Am I wrong? I have read something similar elsewhere.

I had found the page you identified but it meant nothing to me. It doesn't tell me what the valid values are for any of those attributes nor does it contain CAB="2" CCOLOR="Black" CBASE="0" CSTW="-0.1" etc.

Title: Re: .sla file structure
Post by: Nermander on May 31, 2016, 09:52:34 PM
Quote from: Rob Dixon on May 31, 2016, 09:28:17 PM
On the page https://wiki.scribus.net/canvas/Formatxml
it says
"The dimensions of the text frame are based on the live area of the document. That is the size of the page minus the margins.
New pages with connected text frames will be created while the current text frame overflows. "

That is what the script does. The script creates frames based on the live area of the document.

If you look at the code you will see this part:

   def flow(self):
       while(scribus.textOverflows(self.name) > 0 and scribus.getTextLines(self.name)):
           current = self.name
           scribus.newPage(-1)
           scribus.gotoPage( scribus.pageCount() )
           self.name = self.make_textframe()
           scribus.linkTextFrames(current, self.name)


That code checks if the frame overflows or not. If the frame overflows this function is called:
   def make_textframe (self):
       margins = scribus.getPageMargins()
       size = scribus.getPageSize()
       w = size[0] - margins[1] - margins[3]
       h = size[1] - margins[0] - margins[2]
       return scribus.createText (margins[1], margins[0], w, h)


And here you can see that the code creates a text frame.

That script reads an XML file and the script explicitly creates the content in the Scribus document. The script creates text frames, and "types" the content of the XML file into the text frames. The XML is not "imported", it is parsed and handled by the script.
Title: Re: .sla file structure
Post by: Rob Dixon on June 02, 2016, 01:13:53 PM
Nermander,

Many thanks again. If the script that you are referring to is the one from the Formatxml page, then that script didn't work and I was given another by one of the developers which did.

However, I am thinking of trying another approach. Since there does not seem to be any documentation on <ITEXT CAB="2" CCOLOR="Black" CBASE="0" CSTW="-0.1" CSIZE="12" etc., I had decided to create a Scribus document manually, making small incremental changes, saving the document each time, and inspecting the underlying .sla file to see how it has changed.  I won't find all the definitions this way, but I should find enough for my immediate needs. This may take a little while, but, for instance, I have already found that <ITEXT ch="Sample text."/>, without any of the above attributes is how Scribus saves simple standard text, so, for the moment, I don't need to understand the above options.

I think therefore that, rather than generate a .xml file, I should be able to generate a .sla file from my database without too much difficulty. My only concern is the header part of the .sla file, and for the time being I may just copy this into my generated .sla file in Notepad before I open it in Scribus. I will document what I learn and in due course post it if anyone is interested.

I have found that if you create a new document in Scribus with multiple pages and Automatic Text Frames set on, you can copy multiple pages of text from a .doc or other document in one go and paste it into the text frame on the first page, and it will automatically flow the text from page to page, without any problem. I think that this means that I won't have to do my own pagination, unless I want a mandatory page break.

If I generate my own .sla file, I will then be able to open it in Scribus and won't have to worry about importing it. I know that my version of a .sla file will have limitations and, to save time now, I will have to make compromises, but at least it will get me going. Once I have opened my .sla file in Scribus, I can make refinements to the layout in Scribus and then generate the PDF I need.

Are there any obvious flaws in this approach? I have moved to Scribus 1.5.2

Title: Re: .sla file structure
Post by: Nermander on June 02, 2016, 03:08:00 PM
Yes, that might work.

But I think using XSL-FO would be way easier. And you wouldn't have to work against a moving target (the Scribus SLA format will most likely change in the next version).

http://www.codeproject.com/Articles/37663/PDF-Generation-using-XSLFO-and-FOP
Title: Re: .sla file structure
Post by: Rob Dixon on June 02, 2016, 04:07:41 PM
Thanks but I want the intermediate step of being able to refine my document layout with Scribus before creating the PDF. In addition, I don't know Java (only javascript which is vey different) and don't wish to learn unless I really need to. My server is not Windows based but the really robust IBM Power system, running IBM i, and I am not aware that it will run java anyway (although it may do).

I think that my method, limited though it might be, will do all that I want and be much easier to implement that a Java solution, even if I knew that.
Title: Re: .sla file structure
Post by: Rob Dixon on June 02, 2016, 06:39:52 PM
Nermander

A correction - Jave does run under the IBM i operating system - I have never used it
Title: Re: .sla file structure
Post by: Nermander on June 05, 2016, 09:31:33 PM
You can most likely use XLS-FO without using java.

It seems to me that you have not investigated the options.

But if you want to be able to tweak the layout in Scribus after the conversion, why don't you for example convert to HTML or tagged text (using the text import filters that Scribus provides when you use Get text for a text frame)?

Importing tagget text content into Scribus text frames is a totally different issue to "creating a Scribus document from scratc".
Title: Re: .sla file structure
Post by: Rob Dixon on June 05, 2016, 11:59:51 PM
Nermander,

Many thanks. My database does already create virtual web pages by generating HTML and Javascript on the fly. It also can generate XML and PCL 5 for printing. Because essentially the same code is used for all these options, it is relatively easy to generate at least the body of a .sla file and I actually achieved this in part yesterday. If I went on the XLS FO route, I would still be reading the manual. Of course, I still have quite a lot more to do,  but I am not displeased with my progress so far and I think that my chosen route was best for me. It would take me some time to explain the structure of my connectionist database.

In some sense I am importing tagged text into Scribus. You are right to say that I have not considered all the options because I did not know just what they were. You have helped.
Title: Re: .sla file structure
Post by: Nermander on June 06, 2016, 09:33:56 AM
Well, XLS-FO is a "langugae" for attaching formatting to XML data. I thought that was exactly what you was looking for.

To me it seems as if you want to re-invent the wheel over and over. There are already tons of work put down into solutions to handle XML data and convert it to other formats. What would be wrong with trying to re-use that work instead of doing it yourself?

https://en.wikipedia.org/wiki/XSLT
https://en.wikipedia.org/wiki/XSL_Formatting_Objects
https://www.w3.org/TR/css3-page/

Remember that the SLA format will most likely change (again) in the next version of Scribus.
Title: Re: .sla file structure
Post by: Rob Dixon on June 06, 2016, 06:14:35 PM
Nermander

Thanks for persisting with me. I don't want to attach formatting to existing XML data, but to include the formatting as I generate the XML and that, as I have already said, I have in part achieved. Because of the structure of my database, in which meta data is stored integrated with, and provides a context for, the user data, to include, for instance, basic style information in the XML is very simple, particuarly as I am already doing this when I generate HTML and PCL 5, using mostly the same code. I have a run time option that determines whether the XML output is for general use or for including in a .sla format file.