Welcome to NeoOffice developer notes and announcements
NeoOffice
Developer notes and announcements
 
 

This website is an archive and is no longer active
NeoOffice announcements have moved to the NeoOffice News website


Support
· Forums
· NeoOffice Support
· NeoWiki


Announcements
· Twitter @NeoOffice


Downloads
· Download NeoOffice


  
NeoOffice :: View topic - Post-NeoOffice 3.3 development plans: Mac sandbox support
Post-NeoOffice 3.3 development plans: Mac sandbox support
 
   NeoOffice Forum Index -> NeoOffice Development
View previous topic :: View next topic  
Author Message
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Wed Sep 05, 2012 10:07 am    Post subject: Post-NeoOffice 3.3 development plans: Mac sandbox support

Now that NeoOffice 3.3 is out and we know it runs well on Mac OS X 10.8 Mountain Lion, we felt that now is a good time to give an overview of the state of the NeoOffice project and outline our development plans for the rest of this year.

State of the NeoOffice project

All of our development time so far this year has been spent "de-Javafying" NeoOffice and getting NeoOffice ready for Mac OS X 10.8 Mountain Lion. While users probably won't notice anything different from our massive code changes other than some small increases in speed, these tasks were critical.

As a result of our de-Javafication and Mac OS X 10.8 Mountain Lion work, NeoOffice is the only full office suite other than Apple's iWork application that supports Retina displays, Gatekeeper, Versions, Full-Screen, and Core Text.

Sadly, Mac OS X development appears to no longer be a priority for Microsoft Office, OpenOffice.org, and LibreOffice. Currently, all 3 of these office suite applications only support one of the Mac OS X standard features listed above. Specifically, Microsoft Office does not display text correctly on Apple's new Retina displays and users must disable or bypass Apple's new Gatekeeper feature to install and run OpenOffice.org or LibreOffice on Mac OS X 10.8 Mountain Lion.

Development plans for the rest of 2012

In Mac OS X 10.8 Mountain Lion, Apple added an important new security feature called Gatekeeper and we worked very hard to ensure that a fully compliant version of NeoOffice was available when Apple released Mac OS X 10.8 Mountain Lion.

While Apple has not announced any plans for the next version of Mac OS X, our gut feeling is that Apple will continue to add new security requirements for applications. Maybe our guess will prove to be wrong, but we think that Apple will, at some point, require applications to implement Apple's App Sandbox security feature.

Apple has already started requiring applications submitted to the Mac App Store to implement the App Sandbox security feature so we feel that that is a fairly clear signal that Apple would like to eventually see the App Sandbox security feature in all Mac OS X applications.

So for Ed and I, implementing the App Sandbox feature will be our most important development project at least through the end of 2012 and maybe into 2013. We have already started working on this project and we have already found that NeoOffice's underlying OpenOffice.org code must be changed to run with the App Sandbox feature enabled so we know that this project will require a lot of our available development time.

Stability is still our primary goal so while we are working on App Sandbox development, fixing any critical crashing and hanging bugs reported in our Neooffice Support forum will continue to be our highest priority.

We understand that many users wish our development time was focused on adding new application features like Microsoft Office, OpenOffice.org, and LibreOffice. Those applications have added some interesting new features over the last few years, but the reality is that their new features have come at the cost of failing to implement the key new features in the last two Mac OS X versions. For some users, not having the key new features Mac OS X features would make NeoOffice undesirable or even unusable and, as we have found, these features cannot be implemented overnight.

Being focused specifically on Mac OS X, we feel that implementing Apple's new security features are required to maintain the longevity of NeoOffice as an application. In the past decade we have seen many new Apple technologies become common Mac OS X features and so we feel that it is probably inevitable that Apple will adopt many of the security features in iOS in future versions of Mac OS X. By focusing our skills on adopting Apple's latest technologies we feel we are continuing to make the best Mac OS X experience by ensuring that our application is future proofed to the best of our ability.

Edit by pluby: changed title of this forum topic to be more descriptive

Patrick
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Tue Nov 20, 2012 8:18 pm    Post subject:

After working on implementing Apple's App Sandbox security feature for the last 2 months, we have some good news to report: we have gotten NeoOffice to install and run in the App Sandbox.

While we still have a few months of work left to do before we can release a beta version of NeoOffice, we think we now have a pretty good idea of what changes users will see when NeoOffice fully implements the App Sandbox.

Before we go into the changes, please note that none of these changes add features and, instead, Apple's App Sandbox improves security by severely restricting what applications can do. So the question is why are we implementing the App Sandbox?

The answer is that we do not believe that ignoring Apple's App Sandbox is a feasible option for more than another year or two. While the App Sandbox is currently optional, our gut feeling is that increasing Mac OS X security is a high priority for Apple. Our prediction may end up being wrong, but it seems highly likely to us that in the next few versions of Mac OS X, Apple will make it difficult for applications to avoid the App Sandbox.

Edit by pluby: added Base changes to end of the list of changes

Below are the three types of changes that we have identified so far that users will notice in the next version of NeoOffice:

More frequent display of the native open dialog

By default, the App Sandbox severely limits which files and folders an application can open or save without explicit permission from the user. Users give explicit permission by selecting a file or folder in the native open or save dialogs. Without explicit user permission, applications running in the App Sandbox will not be able to access most files and folders where users commonly save documents.

To handle this App Sandbox restriction, we have added code that displays a native open dialog if NeoOffice needs to access a file or folder that requires explicit permission from the user. For example, when a user tries to open a document listed in the File :: Recent Documents menu, a native open dialog like the one shown in the following screen snapshot will appear and the folder that the document is in will be selected. If the user presses the native open dialog's "Open" button, NeoOffice will be able to access any files in the selected folder or its subfolders even after restarting NeoOffice.



Extensions and macros that use Java or Python will no longer run

Extensions or macros written in OpenOffice.org Basic or natively compiled C++ code will continue to work but, unfortunately, any extensions or macros that use the Java or Python programming languages will not work when NeoOffice is running in the App Sandbox.

We understand that this is not good news for many NeoOffice users due to the fact that many developers chose to write their extensions and macros using Java or Python. However, it is just not feasible for us to take on customizing the Java and Python language runtimes to work within Apple's App Sandbox.

Since all extensions and macros that use Java or Python will not work, we are removing all references to Java and Python within NeoOffice. For example, menus that list supported macro language types like in the following screen snapshot will show NeoOffice Basic as the only supported macro language:



Base will not be able to open databases that require Java

By default, Base creates new databases in the HSQLDB database file format. Since there only only Java-based drivers for HSQLDB, users will not be open to open their existing Base databases created in the next version of NeoOffice.

Base will still be able to create new databases as NeoOffice's underlying OpenOffice.org Base code will sense that Java cannot be loaded and will create a database in a plain text file format.

All of the Java-based database drivers (those with "JDBC" in the driver name) shown in the NeoOffice 3.3 (see left screen snapshot below) will no longer be available when NeoOffice is running in the App Sandbox (see right screen snapshot below).



As a result of the lack of Java-based database drivers, users that have existing HSQLDB database files will need to continue using NeoOffice 3.3 or earlier to access those existing databases or, before moving to the next version of NeoOffice, migrate the data in their existing databases to a new database type.

Users who have existing databases that access a MySQL or other remote database have another option: install an ODBC driver for your type of database on your machine and change your database to use the ODBC driver instead of the JDBC driver.


We hope that this post gives users a clearer picture of how we are adapting NeoOffice to work within the constraints of Apple's App Sandbox security feature. We will continue to post updates as our development progresses.

Patrick


Last edited by pluby on Thu Nov 22, 2012 1:18 pm; edited 1 time in total
Back to top
ovvldc
Captain Naiobi


Joined: Sep 13, 2004
Posts: 2352
Location: Zürich, CH

PostPosted: Wed Nov 21, 2012 4:58 am    Post subject:

Congratulations that your work in this is progressing well.

I am wondering about the loss of java for macros. Will this impact the parts of NeoOffice that use Java? I recall Base needs it somewhere, and some wizards.

Best wishes,
Oscar

_________________
"What do you think of Western Civilization?"
"I think it would be a good idea!"
- Mohandas Karamchand Gandhi
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Thu Nov 22, 2012 1:22 pm    Post subject:

ovvldc wrote:
I am wondering about the loss of java for macros. Will this impact the parts of NeoOffice that use Java? I recall Base needs it somewhere, and some wizards.


Good question. I knew that I was missing something when I wrote my last post.

The short answer is that Base will still be there but Java-based database drivers will no longer work. I have added a new section at the end of my previous post that more fully explains the Base changes.

Patrick
Back to top
James3359
The Merovingian


Joined: Jul 05, 2005
Posts: 685
Location: North West England

PostPosted: Sat Nov 24, 2012 10:56 am    Post subject:

I just wanted to echo Oscar's congratulations.

I know it is a way off yet, and this is probably something you already have in mind, but the loss of Java and Python is something you will need to give existing users very clear info about when they are considering upgrading. (Also, I suppose new users who are coming from OOc or LO.)

It may be that it will be worth highlighting the possibility of running both the current and older versions of NO on the same machine, and being clear about the issues involved in doing that.
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Sat Nov 24, 2012 11:32 am    Post subject:

James3359 wrote:
I know it is a way off yet, and this is probably something you already have in mind, but the loss of Java and Python is something you will need to give existing users very clear info about when they are considering upgrading. (Also, I suppose new users who are coming from OOc or LO.)


We are not going to push the next version out as a replacement for NeoOffice 3.3. We will keep maintaining NeoOffice 3.3 and making it work as long as we can.

What we are working towards is a scenario where a new version of Mac OS X makes NeoOffice 3.3, OpenOffice.org, and LibreOffice barely usable or unrunnable. Given OpenOffice.org and LibreOffice's have so far ignored the new Gatekeeper security feature, it is quite possible that in the near future, NeoOffice running in the App Sandbox may be the only option available of the 3.

Since an App Sandbox version of NeoOffice will be a different thing than NeoOffice 3.3, we plan to keep them separate by doing things such as:

- Giving it an entirely different version name like "NeoOffice 2013" or "Secure NeoOffice"

- Emphasizing NeoOffice 3.3 on our website by listing the new version as a beta version and/or trying to put the new version in the Mac App Store.

James3359 wrote:
It may be that it will be worth highlighting the possibility of running both the current and older versions of NO on the same machine, and being clear about the issues involved in doing that.


The App Sandbox forces preference files into a parallel set of folders located deep within your ~/Library/Containers folder so NeoOffice 3.3 and the next version of NeoOffice will have separate, independent preference folders and, as a result, both can be run at the same time without interfering with each other.

Patrick
Back to top
sardisson
Town Crier
Town Crier


Joined: Feb 01, 2004
Posts: 4588

PostPosted: Mon Nov 26, 2012 2:19 pm    Post subject:

James3359 wrote:
I just wanted to echo Oscar's congratulations.

Same here. I'm sure it has been a herculean task to reign in the freewheeling OOo code so that it behaves well enough to be sandboxable yet still actually run!

As I thought at that time, it's a shame that Sun forced Base (or significant parts thereof) to be implemented in Java. It reeked of desperation then, and I'm sorry to see that bad decision come back to bite users now on platforms where Java is on the way out. I hope sometime in the near future either LibreOffice or Apache OpenOffice will rewrite Base to eliminate the Java dependency and NeoOffice can pick up that code once it's been proven stable…

But, to echo James and Oscar, this is really impressive Smile

Smokey

_________________
"[...] whether the duck drinks hot chocolate or coffee is irrelevant." -- ovvldc and sardisson in the NeoWiki
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Mon Nov 26, 2012 6:06 pm    Post subject:

sardisson wrote:
Same here. I'm sure it has been a herculean task to reign in the freewheeling OOo code so that it behaves well enough to be sandboxable yet still actually run!


We definitely still have a lot of work to do. However, the nice thing is that in most cases we will be able to use the same "display a native open dialog" code that we have already implemented with minor tweaks. The timeconsuming part will be tracking down all of the places in the OpenOffice.org code that we need to invoke our new code.

sardisson wrote:
As I thought at that time, it's a shame that Sun forced Base (or significant parts thereof) to be implemented in Java. It reeked of desperation then, and I'm sorry to see that bad decision come back to bite users now on platforms where Java is on the way out. I hope sometime in the near future either LibreOffice or Apache OpenOffice will rewrite Base to eliminate the Java dependency and NeoOffice can pick up that code once it's been proven stable…


Note that except for a few Java-based wizards, Base really is just a graphical user interface for databases that are accessed using JDBC and ODBC database drivers. The problem is that the HSQLDB developers only wrote a JDBC driver and since the "J" in JDBC stands for Java, the core problem is that HSQLDB has no native ODBC driver.

Databases like MySQL and PostgreSQL have had both JDBC and ODBC drivers for a long time, but HSQLDB has only recently started development of an ODBC driver.

Hopefully the HSQLDB developers will eventually have a stable ODBC driver before Apple finally stops making Java available in new Mac OS X versions. However, their ODBC driver code does not appear buildable on Mac OS X and appears to include some Java code so we are planning for the worst case: no Java and no native ODBC driver for HSQLDB.

Patrick
Back to top
amayze
The Merovingian


Joined: Oct 24, 2005
Posts: 561
Location: Edinburgh, Scotland

PostPosted: Tue Nov 27, 2012 2:57 am    Post subject:

Losing JDBC support from HSQLDB will be a major hit for me going forward. I rely on Base and HSQLDB 2 to run part of my business, however since the database currently runs on a mac that is stuck at OS 10.6 there is no reason for me to break things by installing 'NeoOffice 2013'.

I'll need to look in to what to use for future development though. H2 is also a candidate, though it's written in Java it has an ODBC driver. The features table on the H2 website gives a useful summary of what will and won't work with ODBC drivers.

Andy
Back to top
ovvldc
Captain Naiobi


Joined: Sep 13, 2004
Posts: 2352
Location: Zürich, CH

PostPosted: Tue Nov 27, 2012 3:55 am    Post subject:

Would there be any way of compiling the Java code in the wizards into native binaries not dependent on external Java stuff?

I would have suggested it also for the HSQLDB and JDBC driver, but I guess that will be really complicated in any case..

_________________
"What do you think of Western Civilization?"
"I think it would be a good idea!"
- Mohandas Karamchand Gandhi
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Tue Nov 27, 2012 8:12 am    Post subject:

ovvldc wrote:
Would there be any way of compiling the Java code in the wizards into native binaries not dependent on external Java stuff?


You mean do we have time to rewrite all of those wizards native. The answer is no.

If you are dependent on HSQLDB, a JDBC driver, and/or the Java-based Base wizards, just stick with NeoOffice 3.3.

Patrick
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Tue Nov 27, 2012 3:30 pm    Post subject:

amayze wrote:
Losing JDBC support from HSQLDB will be a major hit for me going forward. I rely on Base and HSQLDB 2 to run part of my business, however since the database currently runs on a mac that is stuck at OS 10.6 there is no reason for me to break things by installing 'NeoOffice 2013'.


Ed and I are betting that Apple will not retroactively force App Sandbox in Mac OS X 10.8 Mountain Lion or earlier versions of Mac OS X so we see no immediate end-of-the-line for NeoOffice 3.3 yet.

What we are trying to prepare for is the next version of Mac OS X. When Apple releases a developer preview of the next version, we will know pretty quickly if the unsandboxed NeoOffice 3.3 code will be runnable or not.

Our hope is that the NeoOffice 3.3 code will be runnable. In that case, the App Sandboxed version of NeoOffice - "NeoOffice App Store Edition" - can just be a parallel product for users don't need Java or Python and prefer to buy from the Mac App Store.

The key point of this forum topic is that getting NeoOffice to run in the App Sandbox is a big project so we can't wait for Apple to release a developer preview of the next Mac OS X version before we start. Apple keeps moving and we don't feel comfortable relying on Apple making changes slowly.

Patrick
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Sun Feb 24, 2013 8:18 pm    Post subject:

During the last few months we have been making steady progress on implementing Apple's App Sandbox. Although progress has been slower than expected, we are getting closer to our goal of submitting an "App Store Edition" version of NeoOffice to Apple's Mac App Store.

As I mentioned in this previous post of mine, Apple requires that all applications submitted to the Mac App Store use the App Sandbox and to not use Java. Since these restrictions will cause our planned "NeoOffice App Store Edition" product to be missing some features, we want to reassure everyone the full-featured NeoOffice versions that we distribute via our website is not going away.

The full-featured versions will continue to get bug fixes and new features like now and we will copy those bug fixes and new features to the NeoOffice App Store Edition after they have been in use by regular NeoOffice users. Essentially, the NeoOffice App Store Edition will be a parallel version of NeoOffice that has been modified to fit within Apple's Mac App Store security requirements.

Screen snapshots

Below are a few screen snapshots of some of the changes that we have made since my last post. Since Apple explicitly does not allow Java in Mac App Store applications, we have been removing all Java-based menu items and toolbar buttons. In Writer, Calc, Impress, and Draw documents, we only needed to remove some of the items in the File :: Wizards menu as shown below (the left snapshot is from NeoOffice 3.3 and the right snapshot is from our NeoOffice App Store Edition build):



Unfortunately, in Base we needed to remove all of the table, view, and form wizards so users will need to use the "Design View" option when creating tables, views, or forms. Also, all of Base's reporting functionality requires Java so we needed to remove all reporting menus and toolbar buttons. The screen snapshots below show how Base will change (the left snapshot is from NeoOffice 3.3 and the right snapshot is from our NeoOffice App Store Edition build):



What's next?

We still have some work to do before NeoOffice App Store Edition will be ready for testing. Specifically, we need to implement a way to replace all help entries for all of the Java-based menu items, toolbar buttons, and other functionality that we removed and have those help entries display some standard text such as following:

Quote:
This feature is not available in NeoOffice App Store Edition due to Apple's Mac App Store security requirements. If you need this feature, it is available in the full-featured version of NeoOffice.


Also, when using Apple's App Sandbox, the version browser only displays a small thumbnail image of old versions (see right half of screen snapshot below). Although we filed a technical support incident with Apple back in November 2012, we doubt that Apple will fix this anytime soon so our next task is to investigate if we can override the QuickLook plugin loading process and grab the full QuickLook preview for an old version instead of only its thumbnail image:



Patrick
Back to top
amayze
The Merovingian


Joined: Oct 24, 2005
Posts: 561
Location: Edinburgh, Scotland

PostPosted: Tue Feb 26, 2013 7:46 am    Post subject:

pluby wrote:
Unfortunately, in Base we needed to remove all of the table, view, and form wizards so users will need to use the "Design View" option when creating tables, views, or forms. Also, all of Base's reporting functionality requires Java so we needed to remove all reporting menus and toolbar buttons. The screen snapshots below show how Base will change (the left snapshot is from NeoOffice 3.3 and the right snapshot is from our NeoOffice App Store Edition build):

In light of these changes, would it perhaps not be better (if technically possible) to remove Base entirely from NeoOffice App Store Edition?

If less tech savvy users are more likely to use the App Store than the NeoOffice website for their download, then they are perhaps also less likely to need access to Base's features. If they do try to use it they will find it very difficult, particularly as it will look so unlike what they will see in various online guides to Base.

I can see the current version (above) generating a lot of support requests for you. Perhaps it would be better to present a dialogue directing users to the full version of NeoOffice if they try to open or create an .odb file.

To counter my own argument, could this be done without also breaking form letter support in Writer, or has that already gone too. I have no idea if it also relies on Java to work.

My 2p.

Andy
Back to top
pluby
The Architect
The Architect


Joined: Jun 16, 2003
Posts: 11949

PostPosted: Tue Feb 26, 2013 9:35 am    Post subject:

amayze wrote:
In light of these changes, would it perhaps not be better (if technically possible) to remove Base entirely from NeoOffice App Store Edition?


I don't think that is a good idea. Sure, it is less user friendly for users that try to create a new database, but having it there allows people with existing shared remote MySQL, postgres, or other databases to still use their input forms and queries. Not everyone is using Base for just single-user data; many use it as a GUI front end for input and queries into a workgroup's database.

amayze wrote:
If less tech savvy users are more likely to use the App Store than the NeoOffice website for their download, then they are perhaps also less likely to need access to Base's features....


I am not sure that I agree that only "less tech savvy" users would use NeoOffice App Store Edition. The Mac App Store provides 2 things for Mac OS X users: convenience (you pay by logging in with your iTunes account and increased application security (because Apple impose security requirements before allowing an application in the store).

It is that second item - increased security - that is the reason we have been targeting the Mac App Store. Restricting application installation to only Mac App Store apps is already an option in Mac OS X 10.8 Mountain Lion and it wouldn't surprise me to see that or other application installation restrictions imposed on users whose machine is maintained by their employer.

amayze wrote:
I can see the current version (above) generating a lot of support requests for you. Perhaps it would be better to present a dialogue directing users to the full version of NeoOffice if they try to open or create an .odb file.


What you describe is not allowed in Mac App Store apps as displaying a dialog that sends you to a non-Mac App Store site that sells a product (i.e. our downloads) is forbidden by Apple.

Even if we could do what you suggest, we won't get any support requests from the Mac App Store as Apple does not even let us know who has purchased our product. Essentially, Apple is a retailer and we are the manufacturer for any of our App Store apps: we provide the product and Apple only sends us one payment each month for the net purchases of the product.

Apple does not give access to any customer data to app developers (which is a good privacy feature of the store) so if someone buys NeoOffice App Store Edition to open a HSQLDB-based .odb file or finds that their favorite third-party Java-based or Python-based extension does not install (and they haven't read any of the warnings of limitations that we'll put in our store description), their only options are to live with the limitation is to ask for a refund from Apple. Apple will decide whether or not a refund is justified as they are the retailer. We will have zero contact with the user.

Also, because Apple does not give access to any customer data to app developers, we won't sell support through the Mac App Store. Support will continue to be provided the same as now: only to users paying $100 or more in the last year and, realistically, fixes will go into the full-featured NeoOffice version and then periodically get copied to the App Store Edition.

amayze wrote:
To counter my own argument, could this be done without also breaking form letter support in Writer, or has that already gone too. I have no idea if it also relies on Java to work.


If you remove all Base support, you remove all database support which means that anything that uses a "data source" would be gone.

Patrick
Back to top
Display posts from previous:   
   NeoOffice Forum Index -> NeoOffice Development All times are GMT - 7 Hours
Goto page 1, 2, 3  Next
Page 1 of 3

 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum

Powered by phpBB © 2001, 2005 phpBB Group

All logos and trademarks in this site are property of their respective owner. The comments are property of their posters, all the rest © Planamesa Inc.
NeoOffice is a registered trademark of Planamesa Inc. and may not be used without permission.
PHP-Nuke Copyright © 2005 by Francisco Burzi. This is free software, and you may redistribute it under the GPL. PHP-Nuke comes with absolutely no warranty, for details, see the license.