[BUG] 100% Reproducible data loss of saved service (*.osz) data file (moved from Troubleshooting)

FromOZ

Apologies — posted this first in Troubleshooting then saw that bug reports should go here.

I can confirm this behaviour again. I see that the servicemanager folder seems to be a scratch folder. The obvious key issue is that OpenLP lets the user save service files there.

[=====================================]

In my learning to use OpenLP I lost track of the standard folder where the service files are saved to. Looking around in the settings I saw that the standard data path was 

C:\Users\%USERNAME%\AppData\Roaming\openlp\data

so I saved a test service (*.osz) file in the subfolder

C:\Users\%USERNAME%\AppData\Roaming\openlp\data\servicemanager

because I thought it looked a likely suspect.

When I went back into OpenLP (few minutes later) I found that the service file was gone.  So I made another and saved it in the same folder — same behaviour. Next I saved the file, then with OpenLP still running I confirmed that the .osz file was saved and present in the folder. Then I exited OpenLP and the file immediately disappeared.

I then tested again saving a file in

C:\Users\%USERNAME%\AppData\Roaming\openlp\data

and that one was fine, did not get deleted.

I am not sure of the function of the servicemanager folder but OpenLP happily let me save a service file in there with no warning and consistently deleted the file on exiting the app.

So far I can reproduce this at will - system is Win 7 SP1 running in Parallels on a Mac.

tgc

Hi FromOZ

This is actually not a bug. 

The thing is that you should not save your data in the openlp data folder (C:\Users\%USERNAME%\AppData\Roaming\openlp\). That folder is for OpenLPs internal data storage, and you should not touch the data in those folders unless you know what you are doing, or else you might experience data loss - as you have.

Place your service files in your Document folder or on your Desktop, or anywhere else than in the OpenLP data folder.

FromOZ

Hi tgc

Sorry for delay in posting response - have been busy setting up and migrating our church setup from EasySlides to OpenLP!

To your comment above — I'm afraid I am going to have to disagree with you. If I follow this definition of a software bug:

"A software bug is an error, flaw, failure, or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways."

Then my reported issue is a bug — and furthermore a serious bug because it results in data loss — consistently & without warning. I don't know if you are one of the developers of OpenLP however I believe everyone would agree that — consistent, reproducible and total data loss would not be one of the intended ways of OpenLP working, correct?

There are actually two issues related to this in OpenLP on both Macs & Windows:

1. The 'Data' location in the Advanced screen in configuration gives one the impression that this is the i.e. yours, the users', data folder. When actually, strictly speaking, it is the OpenLP system data folder which should be, typically, off limits to the (average) user. Which is why I am assuming the developers put the folder in usually inaccessible folder locations like app data\roaming on Windows and /Library/Application Support on the Mac. However: 
1. there is no user defined user data folder selection option in OpenLP
2. when users install the software OpenLP has them, by default, save files in the OpenLP system data folder. This design I believe can and should be improved/changed. If OpenLP followed Windows & Mac user conventions then it should create the following folders on install:
1. Windows: ~ My Documents\OpenLP
2. OS X: ~ Documents/OpenLP
2. Regardless of all the above, OpenLP should never delete user data — that is a bug. One could argue that the user should not save files in the service folder, they should know better (how??) but the point is (well my point) the software should protect the user from inadvertent data loss, any attempt by the user to save a service file in the servicemanager folder should be blocked with a data loss warning. I would go even further to say, given the dangerousness of that folder — that OpenLP trashes everything in there on closing down the program — that the software should create that folder on the fly and make it hidden from the user and unwritable by the user.

Basically... I think it is the responsibility of the software to protect user's data, and to protect the user from making data loss mistakes, and to do so in a proactive way.

FromOZ

Let me put it to you this way....

• We would all agree that a user losing all their work is not a 'good thing'.
• We would all agree that OpenLP allows this data loss from within the software without warning.

Now as to whether the user should be 'smart enough' to know that OpenLP is going to destroy their data because they saved it in the 'servicemanager' folder or not is debatable.

What I would like to just add is... imagine if the warning "don't save your service files in the servicemanager folder because the software will delete them without warning" was added to the OpenLP User Guide manual pages (it should be put there, this is, after all, a user data loss topic). Would not people say "boy, that's bad, why don't they fix that instead of putting a note in the user manual".

tgc

Despite your arguments above I'll maintain that this is not a bug. And I'll argue that the users are indeed smart enough to not save files in the OpenLP data folder. The fact that you are the first to complain about this seems to support that view.
That being said OpenLP is an open source project, and I am not the project leader, so if any of the developers agrees with you and are willing to provide patches, your wish might come true.

FromOZ

It surprises me that you seem to be so blasé about behaviour of the software that destroys data files.

Maybe I'm the first to stumble upon this and smart enough to figure it out? As I mentioned in my first post I was testing about five different packages to replace EasySlides, OpenLP being one of them. With OpenLP I installed 2.0.5 then the 2.1.2 beta, uninstalled that and went back to 2.0.5 and in all that back and forward could not see where service files were, by default, meant to be saved (because OpenLP does not make a user data folder). Then I saw the 'data' folder listed in the settings and thought well it does say "data" and it does have a folder called "servicemanager" and I am saving a service file...

I will point out again that the program should follow Windows & OS X user interface guidelines, currently, on the Mac at least, it is not.

https://developer.apple.com/library/mac/documentation/General/Conceptual/MOSXAppProgrammingGuide/AppRuntime/AppRuntime.html

if the software is going to have a temp folder where the contents are to be deleted on program exit then it should use the system temp folder location. To be a well behaved OS X application it should make an OpenLP user data folder under ~/Documents

I would have thought that there would be some gratitude for finding an issue like this, instead I am obliquely referred to as not "indeed smart enough". You'll forgive me if I beg to differ.

PaulR

On windoze the appdata folder is a hidden folder most of Joe Public would not even know it was there.

In my humble opinion I wouldn't be surprised if there are many other programs that have temp directories in subfolders of that folder. 

-Paul

FromOZ

Sigh....

I realise that this is an open source app (with its roots in Linux distros I think) and it's wonderful that there are a team of guys doing this service for everyone. I wouldn't be migrating our church to it if I didn't think it wasn't any good!

The fact that it's free, a labour of love, etc. doesn't mean it can't be improved!

The fact that OpenLP does not follow operating system conventions in both Windows and OS X (making a proper user data folder), using OS conventions of temp folders to use for its own temp file writing is one thing. It would be nice if the app could be enhanced to work more in line with those two OSes.

I'm an IT Director, I know that %APPDATA% and ~/Library are both typically hidden from the average user. Again though the app should make user data directories in the usual places and make it more clear that the system data directory is for the app.

Regardless of all the above... the point I keep on saying is that the application allows - mistakenly or whatever - a user to save a service file in a folder where it will be trashed. 100% of the time. If the developers are happy to have that situation continue... then that's their choice.

tgc

You are of course right in that OpenLP can be improved - our long list of bugs and feature requests can confirm this. These long lists coupled with our limited resource also means that we have to prioritize what is changed or fixed. So sometimes we chose not to do implement certain features or fix certain issues because the potential benefit is too small compared to the resources which will be used. Besides this there also needs to be a developer actually willing/capable of doing the work.

All that being said, I've presented your requests to some of the other developers, and one did mention that he would look into moving the temp-folder to a more standard location. This would most likely remove the servicemanager folder from the data-folder. It is too early to say if/when this will make it into a release. Even with this change it will still be possible for users to save files in the data folders where they might cause trouble or disappear.

FromOZ

Thanks tgc

If I was a heavy duty coder I would offer to help, I know enough coding to be dangerous — but not enough to add any Python code that would win any awards!

Obviously, on a number of levels, I think the OpenLP platform is great. Putting "my money where my mouth is" and being technical I am thinking where I could add some help/support once I get more familiar with it.

Being primarily on the OS X platform I was also wondering about form design and getting the OpenLP UI as close to Mac looking as possible — just interested.

carl

Interesting usability issue/bug but it's not obvious exactly where the fault lies. I see two issues:

1. OpenLP allows you to save service files in the data folder which apparently gets cleaned out regularly by OpenLP (it also lets you save files to other non-OpenLP folders that may or may not get deleted but that's impossible to avoid).
   
2. OpenLP will apparently delete any other files that get copied into that folder (possibly a problem if you change the data folder to a custom location).

Ideally, OpenLP wouldn't delete files without telling you unless it already created them without telling you in the first place and called them something like 123xyz.tmp to make it obvious that they are temporary. This would resolve the issue FromOZ mentioned, if I understood correctly. There should also be a warning against the data folder field in the settings explaining that files in this folder may be deleted; this might just save the guy who sets his data folder to C:\.

FromOZ, I think the reason people were maybe not taking you too seriously was that you did something rather unusual on the face of it, by picking a pretty obscure place to save your file, and possibly one of the only places where it would get deleted. It's a bit like deciding to save your service file in your browser cache folder, or saving an important e-mail in the OpenLP data folder, or telling Word to save a document to C:\Users\myaccount\AppData\Roaming\Microsoft\Office\Recent (who knows what happens to those files): it's possible but quite a bizarre thing to do. However, I can see that the purpose of the OpenLP data folder may not be clear. Certainly OpenLP should never offer to save your service into the data folder by default, that would be asking for trouble.