Images process with a lot of files

score
#1

I’m actually trying to use the Images process as a container for 1182 picture files and display them from a script using the index parameter.

To start with, I have a question : are all the files guaranteed to be loaded by alphabetical order ? This seems to be the case but I’d like to be sure I can rely on this behaviour to use the index parameter.
So thanks in advance for any answer.

Also, I noticed three issues :

  • The only option I have to load the pictures is to click on the “+” button from the Images process’ inspector, go to the pics folder and select all the images inside. The problem is, on osx 10.15 at least, the file selector window that pops up takes a long time to preload the images itself, and if I click on the first file, scroll too fast, and click and the last one while holding shift, not all pictures are imported. If I scroll slooooowly and let every pic preview become ungreyed before clicking on the last one, then it’s ok (the window confirms I have 1182 files selected before I hit the “Open” button).

  • But, even it seems ok, now a more blocking issue appears : the index parameter of the Images process shows a [0;1158] range, where it should be [0;1181]. I just counted and there are exactly 23 files in my pics folder with an extra dot in their name before the final extension, so I guess there’s a file name validation issue somewhere (as I believe dots are valid characters in file names) …

  • The last issue happens when all files are loaded : each time I click on the Images process box, Score freezes for a while (maybe to load all the filenames to display in the inspector) and it’s a little annoying when just arranging boxes positions inside the interval.

But I guess I have an edge case here …
Anyway, the file name validation issue remains for small sets of files.

#2

To start with, I have a question : are all the files guaranteed to be loaded by alphabetical order ? This seems to be the case but I’d like to be sure I can rely on this behaviour to use the index parameter.
So thanks in advance for any answer.

as far as I can see they’ll be loaded in the order in which they are added… if the OS gives them in alphabetical order then that’s what will happen. But score does not enforce it at all ; i can add a “sort” function though if that is useful !

The only option I have to load the pictures is to click on the “+” button from the Images process’ inspector, go to the pics folder and select all the images inside. The problem is, on osx 10.15 at least, the file selector window that pops up takes a long time to preload the images itself, and if I click on the first file, scroll too fast, and click and the last one while holding shift, not all pictures are imported. If I scroll slooooowly and let every pic preview become ungreyed before clicking on the last one, then it’s ok (the window confirms I have 1182 files selected before I hit the “Open” button).

hmm, could maybe being able to drop the whole folder solve that for you ?

But, even it seems ok, now a more blocking issue appears : the index parameter of the Images process shows a [0;1158] range, where it should be [0;1181]. I just counted and there are exactly 23 files in my pics folder with an extra dot in their name before the final extension, so I guess there’s a file name validation issue somewhere (as I believe dots are valid characters in file names) …

indeed, just fixed that one :slight_smile:

The last issue happens when all files are loaded : each time I click on the Images process box, Score freezes for a while (maybe to load all the filenames to display in the inspector) and it’s a little annoying when just arranging boxes positions inside the interval.

huh, weird, I hadn’t noticed it taking time but checking with 1k files to see…

#3

oh no, just found out that it loads the whole image in the inspector to check if the file is valid… fixing that

#4

Fixed, also tried something for the lag on “play”. It should be available at Release Continuous build · ossia/score · GitHub in half an hour (it will have “[images] Upload textures on the fly instead of at the beginning” in the list of commits when it’s done building)

#5

Hi jcelerier,

Just tried the continuous build and I can confirm that the file name validation is working now, I have the right number of pictures available in the index.
I can also confirm that the “frozen UI when moving the box” issue is fixed too.
Thanks !

as far as I can see they’ll be loaded in the order in which they are added… if the OS gives them in alphabetical order then that’s what will happen. But score does not enforce it at all ; i can add a “sort” function though if that is useful !

As we can only control the index parameter from a JavaScript process and don’t have access to the file names, if the files are guaranteed to be sorted in a permanent order, we can have the list of filenames somewhere in the script, sort it likewise into an array, and rely on the array index to display the right pictures. So I think a guaranteed sorting order would be nice to have. But for the moment I can live with the way it works.

hmm, could maybe being able to drop the whole folder solve that for you ?

I think it could for this particular use case, but I can also live without for the moment.

Concerning the lag on play :

The behaviour seems to have changed a bit, but it still occurs : when I open the scenario it seems to load all the files (the lag happens here instead of on scenario start) and when I first hit play there’s no lag, but as soon as I stop and restart, the lag occurs again.

FWIW, I’ve been finishing the ofx alternative I was working on for this particular scenario, and in the end I only keep the file paths in memory, and directly open the files and load them on the fly in a single ofImage instance. For my use case it’s working well (I don’t feel any latency, probably because I have a SSD drive and the images don’t change fast), but it might not be the best solution in all cases.

Cheers