Posted: Mon Nov 29, 2004 2:16 am Post subject: Neo/J and the speed factor
Fabrizio's comment in the J vs no-J thread has raised something I'd like to discuss - Neo/J's speed is an interesting issue. I've set up Neo/J for my friend with an 867MHz powerbook that has only 640Mb of RAM, and it actually runs faster on my 400MHz powerbook with 1Gb of ram!
So we're talking about future goals for the development of Neo/J into NeoOffice 2, and one goal is to port the Ooo v2 once it reaches X11. Is it also a goal to move on up to java 1.4.x eventually? I've got a good reason for asking this...
In my experience, generally when I upgrade a big app (like an office suite) I expect that the newer version will run slower on my machine than the previous version did - new features, more this, more that, more optimised for the new fast g7-and-a-half processors etc. etc.
Now so far, with Ooo, it's been a pleasure, because with each upgrade the app has actually run faster. But those of us who don't have the latest mac or a lot of ram are already experiencing Neo/J as slow compared to Ooo X11 when it comes to navigation, menu display and the actual working environment, even though Neo/J is a lot more usable in terms of fonts, printing and so on.
So I'd like to ask those of you who know about such things: come v2.0, what would be the single thing that would most improve the interface speed? Will the move up to java 1.4.x make the app run faster than with 1.3, or is there some other factor?
I'm asking this because one of the goals that I see for Neo/J development (speaking as a user, not a developer, of course...) is to increase the mac Ooo user-base. Among the people who are going to want to use Neo/J are those of us who can't afford MS Office (leaving aside the "I'd rather die than buy MS" issue ) and those of us who for the same reason can't afford to upgrade to a spanking new mac. So if we're looking at potentially raising funds among the user-base to target development of Neo/J, I suggest that pinpointing factors that would improve speed, and raising funds to make that happen, would allow Neo/J naturally to increase its user-base, as more people would be willing and able to use it. This can't be anything but good - more happy users, a bigger, more supportive user group to take some question-answering pressure off the developers(!), more potential funders to improve the app...
So what are the development factors most pertaining to speed, do you think?
Just to make my position clear, I'm personally very happy using Neo/J as it is, reniced on my slow powerbook - it reminds me not to worship speed! But if it's going to get slower rather than faster, I'd like to invest in that not happening, and I'm sure others would too. This is absolutely not meant to be a bitching session about the app as it stands, it's meant to be asking how we users can help you developers to make it better from everyone's point of view.
I was waiting to see whether anyone else would say anything - but, yes, this is definitely an issue for me, especially since moving from 0.8.2 to 1.1 Alpha.
I'm on an eMac G4, 700mHz, 40GB HDD of which 30GB is available, 640MB RAM, 10.2.5. I do keep several apps open all the time, but Neo/J uses the most memory by far - both observation/experimentation and running top in the terminal confirm that. It is worse for me since I do a lot of work with a very large spreadsheet (150+ worksheets, 650K when saved as .sxc, or 2MB+ as .xls). I keep this particular spreadsheet open all the time, since a spreadsheet that size takes me 3-4 minutes either to open or to save when it's an .sxc (though substantially less time as an .xls).
This is a problem with OO.o as well, but on the Neo/J side, I can confirm that it is far more of a problem since going to 1.1. MUCH thrashing when switching to Neo/J, and also when switching to another app after using Neo/J.
And to answer yoxi's question: I have no idea what can/should be done to improve the speed - most of my programming skill is in BASIC! But again, I can say that something involving 1.1 has slowed things down dramatically for me. I suspect the Aqua menus - just clicking on "File" now can give me the beach ball for a full 30 seconds before the menu drops down - but I'm not positive.
None of this is intended as criticism of Ed or Patrick or anyone else. I am VERY grateful for their efforts. I just wanted to file the report - glad to know it's not just me!
I just want to have another crack at making my position clear, too... I am blessed with a powerbook that's not particularly brawny of MHz, but has a Gb of ram, and I find Neo/J perfectly adequate - perhaps partly because the relative slowness of my cpu has accustomed me to nothing happening with great haste. I have no complaints, personally. But I'm taking the altruistic approach in this thread, or at least that was my intention in my sprawling post above
My interest here is to facilitate the 'inclusivist' development of Neo/J by airing ideas among the developers/users about what steps could be taken to make/keep Neo/J as useable as possible by the broadest possible range of mac owners out there, specifically addressing the speed issue.
My idea was that if it can be made clearer what the issues are that affect speed, and what resources would be needed to tackle those issues successfully (where that's actually possible), then those of us in a position to provide for those resources could do so, and would (more to the point) probably be motivated to do so for the general benefit.
This is really meant to be an adjunct to the discussions going on about fundraising for the development of Neo/J, and targetting specific development issues to direct funds towards. Plus, myself and other users are curious about it, as are many potential users lurking around the edges of Neo/J's campfire... or something...
I have thinking about this performance problem for quite some time and here are my best guesses of where the major performance problems are:
1. Java's memory usage - Not only does loading Java require about 150 MB of virtual memory, but also due to the way Apple implemented Java, Java will preallocate virtual memory even if it is never used.
2. Multiple event loop - When I was implementing drag-and-drop earlier this week, I ran into this problem again. I found a work around, but it reminded me that Neo/J is bottlenecked by that fact that there are 3 event loop threads running at the same time:
- The native (i.e. Carbon) event loop
- The Java event loop
- The OOo event loop
Not only is a lot of processing time consumed in the propagation of an event from one even loop to the next (Carbon -> Java -> OOo), but bottlenecks occur because of the inter-thread locking that occurs to prevent the threads from bumping into each other.
Unfortunately, problem #1 was not fixed in Java 1.4.1 so upgrading the Java version won't help solve that problem.
Solving problem #2, or at least reducing the number of event loops from 3 to 2 might be feasible. Having only 2 event loops would more closely approximate how OOo X11 is implemented. I haven't figured out how I would approach this yet, but I will continue thinking about it.
So if I understand this, the basic deal with java 1.3 & 1.4 is that it'll slow down Neo/J if you don't have a lot of ram, because there'll be a lot of virtual memory page-switching going on.
I'll dig around on some Tiger Prequel forums and see if anyone knows whether java 1.5-to-be has a smaller footprint, or the option for one - though I guess a Neo/J move up to 1.5 would be way, way off ahead, and would exclude Jaguar users (and that's assuming there's a 1.5 implementation for Panther too - one for Jaguar would be too much to expect).
So it sounds like the 'multiple event loops' issue is the practical working-ground for Neo/J speed improvements for the present. Is that something you're interested in targeting in the near future, and if so, how much time do you think it would take up? If you could speculate broadly on man-hours, perhaps I can organise a virtual bake sale on this forum to fund it? I'm sure that if I started a thread saying "Patrick & Ed will be able to speed up Neo/J if we contribute $xyz between us - go for it, folks!" something good would happen...
So if I understand this, the basic deal with java 1.3 & 1.4 is that it'll slow down Neo/J if you don't have a lot of ram, because there'll be a lot of virtual memory page-switching going on.
I can confirm this from experience. When I had 256 MB, NeoOffice/J was giving me spinwheels all the time. Now that I have 640 MB, I rarely see them. Disk thrashing is proportionally less.
I guess my eyes are on Apple in that they fix the RAM usage in Java 5 and speed it up a bit. After all, they *do* get paid for it (and I recall Apple's profits jumping high - put that to good use guys).. .
Joined: May 25, 2003 Posts: 4752 Location: Santa Barbara, CA
Posted: Wed Dec 01, 2004 11:20 pm Post subject:
Caveat: OOo is a memory hog on every platform. Memory usage is frequently pointed out as one of the most fatal flaws of OOo on Linux and Win32. Folks on Solaris don't seem to complain as they're just happy to have any office suite whatsoever
On my machine at work I have 256 MB of ram (definitely paltry compared to my home boxen...) and in my usage NeoJ is on par speedwise and memorywise with the old OOo 1.0.3. To be fair, however, I am running a lot of apps at the same time. CodeWarrior and XCode bleed my memory dry, so either OOo or NeoJ don't have much free physical memory when I'm trying to launch or use either.
That said, on the same box things seemed better running NeoJ on 10.2 then 10.3. 10.3 seems to have increased the memory footprint a bit. My hard drive has also gotten a bit more fragmented in that time, so swapping is just slower.
I also wouldn't expect 10.4 to improve the situation...even if Java 1.5 improves memory usage for Java apps, I'd expect the OS overall to require a bit more memory then 10.3 (or at least the same amount).
In general, I don't think there's any real solution for the memory overhead since even if we can improve it by going to a newer Java, the newer OS will probably want more and OOo itself will always want a bundle. Not sure if that makes sense or not.
Do you think we should add a "recommended" requirements to the NeoJ download pages? Right now there's a minimum section, but not a recommended section. We could list mucho memory recommended
Oblivious I'm talking from the user point of view. I used staroffice for two year and after openoffice for a year (more or less) on a windows nt machine, and I felt fine with those software. Oo was quite fast on a 800mhz pentium II machine with 512mb od ram, and quite stable.
Neooffice is more hungry of ram and, under my pb 12'' 1,33 768Mb of ram, is slower that the windows version. So, I can understand people asking for a complete native version of Oo for macosx, 'cause actually neooffice is for the rest of the rest of us, if you can understand me.
But I can understand more the pluby and ed programming problems, nobody is paying for them work, so they have to use the tools to reach faster the goal of a working Oo porting on macosx.
I agree with ed, somewhere must be witten that neooffice works 'fine' with new macintosh with >1ghz and >640mb of ram, or something similar. I used neooffice on my pb with 256mb for some week and it was useless. Switching from neo to another application (like finder) took about 30 seconds of lag.
I hope pluby could find a way to put a "turbo" in neooffice, not only to have a larger number of users, but also for a more 'prompt' office suite. I fear more of the lag is coming from java machine, so I know there are not miracle in the road map.
Ending, thanks again to pluby and ed for the works they are doing. My criticism is only to show a user point of view.
But without neooffice/j, now I'll be using a six fans windows nt server, instead my quiet pb!
Mmm, this is one of the things I've been wondering about for the future, because so far the v1.9x snapshots I've been trying out on our office win98 machine (in my lunchbreaks, of course...) are noticably slower than the 1.1.3 that's on there. Of course, as a pre-release there are probably lots of bugs still going on, and perhaps debugging code all over the place slowing it down as well, but I do hope that v2 doesn't end up slower on all platforms just because it has more features and database support etc.
Thing is, if there's nothing to be done about it, I either have to buy a faster mac or cultivate some acceptance. And either of those is a good thing, right?
I do hope that v2 doesn't end up slower on all platforms just because it has more features and database support etc.
The thing I've noticed in other large open source projects (read Mozilla) is that by the time you streamline the software so that it's lightening fast, computers are so fast that the improvement isn't even noticed, and the fact that it was slow in the first place isn't an issue any more.
I say get it working 1st, and get it working fast 2nd
Joined: May 25, 2003 Posts: 4752 Location: Santa Barbara, CA
Posted: Thu Dec 02, 2004 11:21 pm Post subject:
That is a fair point. OOo 2.0 snapshots, in my local testing, have been more memory hungry then 1.1...
The problem is really rooted in the OOo source code itself. It is a poster child for C++ template code bloat. There are some templates in the code that, when expanded, have over 100 template arguments. I shit you not.
For non-C++ developers...I've personally never written a template that's needed more then 6, and at 6 arguments I thought it was too much.
More recent perspective...the 1.1 "speedups" weren't actually caused by a reduction in the memory overhead of OOo itself. The StarDiv guys came up with some clever memory mapping to help "preload" the libraries on Solaris and Linux. This preloading didn't quite work as well on Mac OS X. The memory footprint wasn't reduced from 1.0 at all, but was just cleverly worked around. Kudos for the solution, but the problem still remains.
Historical perspective on C++ template bloat...the Apple gcc compiler choked due to the number of template arguments. To get OOo to compile in 638C we had to manually expand them to work around the compiler ICE.
2. Multiple event loop - When I was implementing drag-and-drop earlier this week, I ran into this problem again. I found a work around, but it reminded me that Neo/J is bottlenecked by that fact that there are 3 event loop threads running at the same time:
- The native (i.e. Carbon) event loop
- The Java event loop
- The OOo event loop
Not only is a lot of processing time consumed in the propagation of an event from one even loop to the next (Carbon -> Java -> OOo), but bottlenecks occur because of the inter-thread locking that occurs to prevent the threads from bumping into each other.
Just passing through and this issue interested me. I don't even own a mac at present, so am by no means an expert.
Why is Carbon utilised? I am assuming that it was needed to allow for alpha shading and correct "Mac look and feel" for the menu systems. As is suggested, this would place additional burden on the processing of the application.
which seems to suggest that it is possible to support mac menus through java without utilising Carbon. Without knowing the inner workings of either the mac JVM or Carbon I am not sure this is on the right track, but could this feature potentially reduce the number of required event loops?
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You cannot download files in this forum
) and those of us who for the same reason can't afford to upgrade to a spanking new mac. So if we're looking at potentially raising funds among the user-base to target development of Neo/J, I suggest that pinpointing factors that would improve speed, and raising funds to make that happen, would allow Neo/J naturally to increase its user-base, as more people would be willing and able to use it. This can't be anything but good - more happy users, a bigger, more supportive user group to take some question-answering pressure off the developers(!), more potential funders to improve the app...
