- About
- Community
- Development
- Documents
Haiku, Inc. accepts contract relating to package management
News posted on Mon, 2010-12-27 23:50
contracting, donations, haiku inc., hiring
As seen with the poll for the must-have features for R1, the lack of proper package management is clearly one of the items that are delaying the release of R1. More importantly though, as package management is an actual lacking feature (as opposed to a bug in an existing implementation), even the beta cycle is being blocked as a result. In order to progress, Haiku needs package management.
Despite some brain storming, an initial implementation of a PackageFSand the specification of its package format, little else has been done and much remains.
To help remedy this, Oliver Tappe has submitted a contract for 160 hours for a total of $2,622 USD. His efforts will center around doing the groundwork of getting package management actually started.
Specifically, this involves,
- evaluating various existing package management components, to determine if any make sense as part of Haiku's package management system
- evaluating the strengths and weaknesses of existing meta packages (e.g., apt, pacman, smart, ...,)
- evaluating if (and how) PackageFS can be used in conjunction with any of those meta packagers
These tasks could be seen primarily as a research based contract. While this is not a direct coding task, it is part of the development process. More importantly, is it work that is needed for R1 and this contract will allow it to receive some much needed attention.
The year 2010 has been a milestone in the project. The level of financial support reaching the project, (primarily as donations to Haiku, Inc., as well as to Haikuware Bounties, and the Haiku Support Association) has simply been overwhelmingly positive and encouraging. All of these contributions allow Haiku to be put on the fast track to R1. Thanks to all of you for showing your support!
If you appreciate this contract and would like to see more, please consider a donation to Haiku, Inc., as this will help to support the next contract. As always, Haiku, Inc. is eager to consider contracts for developing Haiku, even for non-C/C++ tasks that help pave the way towards R1.
- Login or register to post comments
Comments
Re: Haiku, Inc. accepts contract relating to package management
Very nice.
I think the dependency management part of maven can be used for some inspiration.
maven.apache.org/
maven.apache.org/guides/introduction/introduction-to-dependency-m...
Re: Haiku, Inc. accepts contract relating to package management
Nice to see this project, and mention of regular financial contributors.
Regarding package management, it may be a good time to also think about how Haiku will update itself. i.e perhaps any possible updates to the final release could be installed by the package manager as well? So, maybe the package manager has to be able to restart Haiku after installations and so on.
Re: Haiku, Inc. accepts contract relating to package management
Nice to see this coming along! Looking forward to package management (and self updating, of course.)
Re: Haiku, Inc. accepts contract relating to package management
Personally I would like to see the following in a package manager...
* A small generic .pkg style installer
* XML based installation specifications internally
* tar xz compression (as it's so good and is already supported in Haiku)
And some nice-to-haves:
* Internet update support? (eg XML can specify an update URL for easy updating)
And the please no's:
* Dependency tracking.
- If the software will not be in a central repo, dependency hell is *easy* to get into "Gutenprint vs Guten Print", etc ... just look at what rpm's did to RedHat before yum/rhn.
Re: Haiku, Inc. accepts contract relating to package management
Before yum? I've used many Fedora releases and they have all used yum. Last time I checked it was still way annoying, with double entries, dependency hell and all that. That's one reason I still don't use rpm-based distributions, when I feel like using Linux that is.
Re: Haiku, Inc. accepts contract relating to package management
Thanks for helping my point :)
I'd rather just install the package and get a "libSDL missing" shared library error, most pieces of software use only one to two special case external dependencies anyway.
Re: Haiku, Inc. accepts contract relating to package management
Why are people so stubborn?
The only reason Yum has sucked (not so much in the past two years) is because of some pretty serious performance issues. I really hope we can have something that is much quicker :D
Try a relatively recent (~2 years) version Fedora or CentOS and you will find that everything you will try to install will just work! No dependency hell. Get with the times.
Dependency hell that was part of Red Hat based distros was not due to the package format, RPM tools or Yum as damn near every "zomg Linux/RPM/Yum/RedHat sucks!" fool loves to imply. RPM dependency hell was caused by LACK OF ORGANISATION of package repositories or people trying to install packages not meant for the target distro or release. A package manager is no magical bridge between distros/releases either.
Truth be told RPM could actually be a drop-in solution for Haiku because it is an agnostic format; especially when compared to the Deb format which means you *must* always stay Debian distribution compliant. RPM does not mean that Linux packages will work in Haiku and it doesn't even necessarily mean that packages will work between Haiku distros. RPM doesn't even imply the .rpm extension and it definitely doesn't mean that we have to use Yum (Smart seemed quite good last I used it).
If you are trying to install a package not intended for your distribution or release you get what you deserve! And if you run into dependency hell while installing from your distribution's repos then no package format or management tools will save you anyway. Go chase after packagers and repo managers with pitchforks and stakes to get the issue resolved.
As far as the "lets not do dependency management" argument goes - I've seen how painful it is to install dependencies for programming projects and for the majority of Open Source applications under OSX. While I appreciate aspects of this approach lets not forget that we must make Haiku very Open Source and Developer friendly.
I have been playing with Pacman lately and I quite enjoy it. It lacks package signing which is a pretty serious security issue. Last I heard they had come up with some fixes for the issue... you can read the bug report yourself. https://bugs.archlinux.org/task/5331
Re: Haiku, Inc. accepts contract relating to package management
To be honest, I don't know the exact technical details as to why it won't work as it should. I have heard several times that "Now it should work because we have yum!" or "Now it should work because we have PackageKit!" or "Now it should work because we have better organization!" and so on, but it never does. I don't know what is wrong with these distros in relation to package management, but it's definitely something. I've tried a lot of Fedora releases up until like a year ago. I've tried PCLinuxOS and Mandriva as well. Every time it didn't take long before I ran into these issues.
I always avoid packages for other distributions, I wouldn't complain about that.
Re: Haiku, Inc. accepts contract relating to package management
I'm not sure Haiku needs Linux-style 'package management'. Haiku definitely needs a user-friendly way of distributing, installing, updating and uninstalling software, though, and it needs to be carefully thought through.
Reading up on PackageFS, it sounds great and is really well thought-out. I think compressed packages are extremely desirable, but would require some thinking through. Compressed packages could be decompressed and cached on the fly in a way that's completely transparent to the end user (i.e. 5gb cache space, packages and/or resources within them are extracted on the fly, when cache space runs out it removes old and infrequently used packages, and when there's insufficient hard-disk space the compressed package can be ran in memory with a slow-down warning). In the long-term, an application-exposed caching API may also be useful, so resources can be pre-emptively cached before use (think: games loading resources for the next level). Manual compresson/decompression could be included as well. To make all this work and be quick, the package would probably need to contain integrity checks for both the compressed and uncompressed versions, and possibly individual files too. Package signing, which would be a very nice security feature, would probably need to do that as well. Updates, which would presumably use something like bsdiff to be efficient, may also be messy.
My only slight concern is how software that is typically packaged together will work with a single entry-point. This applies to both GUI applications and command-line software - vi/vim and Office packages are probably the most simple examples.
I'm not convinced a Linux-style package manager with a centralised repository and automatic dependency resolution is necessarily a good idea.
Native software should largely be reliant on Haiku-provided libraries which have a stable API. By making dependency resolution simple and fast, I can see it becoming a crutch that software developers take for granted, with the end result being the ridiculous cross-dependency problems that Linux software has and the bloat that comes along with it.
I also think it's important that any solution does not put off commercial software developers by favouring open source too much. One of the reasons I like Haiku is that its developers are friendly to all software, open source or not.
Re: Haiku, Inc. accepts contract relating to package management
I think we should make it simple; let the package manager extract a self-contained application into /boot/apps/APPNAME, with libraries in /boot/apps/APPNAME/lib. Libraries will then not pollute the system, and the apps will be portable and work right away. The focus on native-ness and not relying on too many external libs will be natural, as the app will get larger the less native it is. Anything else gives me the Linux chills.
I also strongly believe we need to keep the core system separate from external applications. Even if you remove every single application from the package manager, the system should still be working.
Re: Haiku, Inc. accepts contract relating to package management
I don't understand why you prefer directories to packages/bundles. Applications being standalone packages/bundles seem hugely beneficial, even with the problem of multiple entry-points and the complication of compression (although, having thought about it, simply distributing .hpkg.gz and forcing people to decompress may be the best compromise in that regard).
I agree with you about the Linux comparisons and how dependencies should be handled, though.
Re: Haiku, Inc. accepts contract relating to package management
I don't know if I understood you correctly now, but are you thinking that we should have something like an APPNAME.app file inside /boot/apps (by default of course; since it's portable you can move it if you want), which functions like a renamed archive in which everything belonging to that app resides, instead of giving it its own directory? I can definitely see some elegance in that, if it's not overly complex and if it means that you just double-click it to automatically open the default executable, and you can somehow browse and edit the contents by right-clicking -> Explore or something to that extent.
Re: Haiku, Inc. accepts contract relating to package management
"but are you thinking that we should have something like an APPNAME.app file inside /boot/apps (by default of course; since it's portable you can move it if you want), which functions like a renamed archive in which everything belonging to that app resides, instead of giving it its own directory?"
Yes, as per OS X - this really is the most elegant solution. Decompress an application, double click it, and it works. This is exactly what they do. Could make universal binaries containing GCC2/4 binaries & libraries. There really isn't a need for a package manager. OS X proved that.
Re: Haiku, Inc. accepts contract relating to package management
Sounds good. But OS X has its Unix legacy with a Unix directory structure, which AFAIK they try to work around through a lot of trickery making the user see a directory structure that doesn't exist instead of what's really there. I would like to leave that part of the OS X experience out. Otherwise it's a good idea.
Universal binaries? I don't think we even need that. We could just have two binaries in the "archive" (APPNAME.app); a GCC2 one (APPNAME.gcc2.hexe (hexe for Haiku Executable :P) and a GCC4 one (APPNAME.gcc4.hexe), inside, and let the OS automatically run a default binary when you double-click the "archive" (APPNAME.app).
Re: Haiku, Inc. accepts contract relating to package management
Lots of comments here about APT, YUM, Ports etc. What we should really be looking at is iTunes, specifically the Application store.
a) it allows users to rate / comment applications
b) it allows customers to purchase software
Re: Haiku, Inc. accepts contract relating to package management
Never used iTunes, but letting users rate/comment applications and purchase them sounds like good features. How does it work, apart from that?
Re: Haiku, Inc. accepts contract relating to package management
The important thing, IMHO, is to leave the possibility to download the "bundle" from a site and the clicking on it if it needs some dependency let "Package manager" solve it... in this way we solve a great Linux problem if it isn't on the repo... we HAVE to compile it!
Maybe not all the haiku application will be on the Haiku repo, so we have to download it from the developer site...
Another important thing is to avoid the multitude of repo: this app in the Universe repo, this is in the Multiverse, this has a repo only for it (!)... should be one only centralized repo, mirrored as you want but must be the ONE!
I rest on the opinion that the APPLE way was better we NEED a universal means of installation, right, but all the burden of dependency hell?
Well I hope we do in the right way :-)
Re: Haiku, Inc. accepts contract relating to package management
All this talk about dependency this and dependency that, is getting really tiring. Why do we have to make it so complicated? Just make a subdirectory called "lib" for every application and include every library that is needed, inside that "lib" subdirectory before distributing the app, and be done with it. No black magic needed, the user can just run it after extraction with no fuss. No polluting the system. A package manager shouldn't have to do anything other than installing and uninstalling applications IMO.
Re: Haiku, Inc. accepts contract relating to package management
I've been on OS X for about 4-5 years, and download and install a lot of apps. I like going to websites and downloading exactly what I need. With a package manager, I feel like I'm losing control of what's being installed/uninstalled on my system to resolve dependencies. Honestly, I've never had an application fail to start (crash maybe, but rarely), or say it's missing x,y library on OS X, and applications that are 4-5 years old still work because of the way they're packaged. You'll be lucky if an application from BeOS or even an app compiled under Haiku Alpha 2 still works on current revisions.
I think the primary focus should have been on an IDE and guidelines to properly develop applications so that the user doesn't experience non-functioning applications or 'application is missing x,y'. This is probably what lead people to vote for a package manager in the first place. I still find the whole GCC2/GCC4 issue confusing. Which should a developer develop for? What are the advantages/disadvantages of both? Is there an easy way to develop both a GCC2 and 4 binary? Is there a way to package apps so that my binary will run on both systems?
A secondary focus may have been a package manager for OS updates and system files/libraries only. Haiku may be opening a can of worms if they're to accept third party applications into their package manager. Who will administer the publication of apps to the repository, how are apps accepted/rejected, where will the repository be located? etc. I think the intention of this package manager should be clarified as to what content it will provide users.
Re: Haiku, Inc. accepts contract relating to package management
I think binary incompatibility (dependency hell is a different issue) is mostly a thing of the current times. Haiku started out with the focus of being a complete, binary-compatible, continuation of BeOS. The focus on staying compatible with BeOS was dropped before the compatibility had been perfected. Haiku-specific applications aren't supposed to be binary future-proof at this point AFAIK because the ABI is not stabilized yet. I could be wrong, but I think that's where the incompatibility issue lies at this point.
When it comes to the third-party application issue, perhaps we could let every application author have their own repository with only their software. Like let's say Firefox gets ported, then Mozilla will have a Firefox repository run by them. It would never work for Linux because of all the distros, but Haiku is just one OS.
Re: Haiku, Inc. accepts contract relating to package management
Well, this is about personal preference but I don't like the idea of every package bundling libraries, which are then duplicated across tons of other packages and likely the system aswell. If the package is closed source and relies on a particular version of a library then by all means statically link that library with the app.
kvdman:
-"I've been on OS X for about 4-5 years, and download and install a lot of apps. I like going to websites and downloading exactly what I need. With a package manager, I feel like I'm losing control of what's being installed/uninstalled on my system to resolve dependencies."
It's the exact opposite. The package manager logs everything that's installed so that you can actually remove EVERYTHING that was installed. It will also warn you when uninstalling if some other package relies on components of the package you are uninstalling. When you are installing using an included installer (as is often the case on Windows) you really have no control of what is installed, sure the application will show up where you point the installer but you don't know what else/where else it has installed. And as pretty much everyone who has used Windows for a long period of time knows, when uninstalling Windows software, VERY seldom does the uninstaller actually remove everything that was installed, which lead to the classic 'ever-growing Windows' problem.
Then there are programs which comes without installers, apart from using a package manager these are my favourite type of distributions, just unpack to a directory. Simple to remove when you no longer want it. Problem here is external dependancies such as libraries, on Haikuware there are alot of ported apps that needs libraries from those huge library packs, so you install huge library pack A. Sweet, now the application works. Then you install another app which needs a library thats not available in pack A, so you install huge library pack B. Now the application works, but your system is filled with a ton of libraries that you don't need. This is not the light efficient model with which I associate Haiku. A package manager makes sure you only install the libraries you need for the applications you have installed. It also allows you to safely remove uneeded libraries together with applications. I don't buy the whole 'diskspace is cheap' excuse, and again it's far from what I envision a Haiku system to be about.
Now I can understand that Karl (kvdman) isn't to keen on a package manager since it will make Haikuware less needed, but I think Haikuware and a package manager can complement eachother very well since I think an official package manager will focus mainly on larger/stable packages and system/library upgrades.
Finally, there's no right or wrong here, we all have our preferences. I find a package manager to be a great way of keeping a lean updated system with only the files I actually need. Your mileage may wary.
Re: Haiku, Inc. accepts contract relating to package management
"Now I can understand that Karl (kvdman) isn't to keen on a package manager since it will make Haikuware less needed"
That's not the reason. I just don't think a package manager will solve Haiku's problems, and a website can convey much more information. The way software is developed on Haiku needs to be sorted out first. BeOS wasn't released without an IDE, an easy way for developers to program for their operating system and relevant documentation - this to me is more important. Limited package management on BeOS actually came later through a third party commercial entity called Software Valet (where the pkg format came from). Although BeOS had their own software (BeDepot I believe), it was a failure, and they realized it was best developed and handled by a third party.
I think most purists will always turn to websites for their software needs. The majority of the world (i.e Windows & OS X users; and even BeOS users) are, and were used to fetching their software from website repositories (download.com, versiontracker, and bebits) - this is the market and users Haiku will inherit. As you said, it's all a matter of preference. I find this type of software will only be useful to technical Linux minded folks (or for core OS updates), and conversely, the majority of users will use websites for software. I hate to say it, but after thinking a bit about it, I find this a waste of precious resources. That said, I have contemplated pleasing everyone and have looked into package management software for haikuware myself - it's not my preference, priority, or other people's resources though.
A final note, the statistics for the 'final features poll' didn't meet justification for this contract and the priorities voters had. I outline my thoughts here:
haikuware.com/20101231558/final-poll-features-analysis
Re: Haiku, Inc. accepts contract relating to package management
I am with KVDman here.
Package management solves a problem that simply should not exist. This problem DOES NOT EXIST, unless the Haiku community and the Haiku devs allow it to exist.
I herby forward a much better solution.
I would this solution
HCT
Haiku Compliance Testing/certification "or something similar" create a logo to. Nothing fancy.
The basic idea here is that Haiku developers institue a way for application builder to create software, and enforce the standard. All APPS that meets those standards get a HCT symbol for thier app.
What does this mean ?
Well it means that third party software must not corrupt or alter in anyway core OS files "which I would recomend finding some way to secure". Multiuser modes is not needed either, just a file password is enough to alert the user.
It means that each application must be tested to ensure it properly utilizes