How many fields is too many for a scribusgenerator .csv?

Previous topic - Next topic

GnKnG

Well I was curious - will it fail at say 255 fields or at maybe 32kb row size perhaps ??? ...and to answer myself:

IDK- but quite a lot!!! :) > over 2000 fields with fairly verbose headers, and at least 32k per row ....

thus a well prepared mega-csv (where constructing the header becomes a non-trivial logistical text-challenge) can be used to pick and populate from large amounts of mixed data/content directly into graphically editable templates 'direct' from database-generated .csv- very nice job indeed Scribus + Generator team:) 


dragonfly

#1
I drew the same conclusion when first trying ScribusGenerator. It can be used in multiple levels of abstraction where the VAR table in csv can hold links to large text blocks held in backend database. Thus custom documents can be created "on the fly".  Can also link to GoogleSheets  through Python API.
There is another approach I tried. It is to use VAR notation to embed PHP snippets and the document (as XML with PHP clips embedded by ScribusGenerator) is then parsed in PHP server, to spit out XML>SLA file.

GnKnG

 :D - good thinking - i have just been musing on a layered approach for text/image/caption heavy documents > maybe one script first sets out VARs for a second script ... or maybe a final single line (looooong row) from which final alterations can be pulled from > i expect performance to become pretty poor with images, but using small/medium text blocks  speed of .sla generation seems impressive.

berteh

Hello @GnKnG , @dragonfly ,

The core of ScribusGenerator is a plain text parsing and replace mechanism to generate the SLA documents, even when you use dynamic images and other advanced attributes.

The only limits I reached were in the size of the initial scribus template being too big (eg if you include heavy images inline)... but as soon as the template is efficient/light "enough" to be parsed at once in your RAM you're good to go above 1M csv entries ;)


Generating the PDF files on the other hand is all on Scribus... and that may take quite a while even for just a few pages (you're -kindly- warned ;)