Wie viele Nutzer in diesem Forum sicher wissen, ist das Management von Binärpaketen unter FreeBSD derzeit suboptimal. Es hat viele Gründe, aber hauptsächlich zu nennen sind dort 2:
- Die Paketdatenbank in /var/db/pkg speichert nicht alle notwendigen Informationen, sie ist langsam, ineffizient und fehleranfällig.
- Die im Basissystem enthaltenen pkg_* Tools sind ebenso fehlerhaft und bieten nicht genug Funktionen.
In der Vergangenheit gab es mehrere Versuche um beide Nachteile herumzufrickeln. Reine Port-Tools wie Portmaster wurden erweitert, um auch Binärpakete verwalten zu können. Portupgrade implementierte seine eigene Datenbank. Dennoch erreicht man nie die Klasse, die Nutzer voin Linuxdistributionen gewohnt sind. Aus diesem Grund wurde in den letzten Monaten der Binärpaketmanager "pkgng" geschrieben. Die Designziele sind:
- Ersetze /var/db/pkg durch eine neue Datenbank, die schneller und robuster als die derzeitige Struktur ist. Die Wahl fiel auf eine sqlite-Datenbank.
- Erlaube weitere Metadaten und verändere das Paketformat so, dass es diese Metadaten speichern kann. Diese Metadaten werden nun in Form von YAML-Dateien gespeichert, denn dieses Format ist problemlos von Menschen lesbar und auch maschinell leicht zu handhaben.
- Entkoppele das Basissystem von den Ports. So wie ich es verstanden habe, soll pkgng nicht mehr Teil des Basisystems werden, viel mehr ein integraler Bestandteil der Ports. Es muss im Rahmen der Installation des Portstree installiert werden. Vielleicht kann hier noch jemand etwas zu sagen, der mit der Materie vertrauter ist.
- Stelle ein Interface bereit, was sich an gängigen Linux-Paketmanagern orientiert. pkgng ist daher ein Hybrid geworden, der Funktionen aus apt, yum, packman und einigen anderen Tools aus der Linuxwelt kombiniert.
Die unten stehende E-Mail erläutert pkgng noch ein wenig genauer und gibt Hinweise zu Benutzung des Tools. Es ist geplant, pkgng 1.0 gemeinsam mit FreeBSD 9.1 auszuliefern, was für den Sommer geplant ist. BITTE BEACHTET: Ist die Die Paketdatenbank muss konvertiert werden! Die E-Mail sagt nichts dazu, aber es scheint, dass der Weg zurück zu den bisherigen pkg_* Tools damit versperrt wird!
- Die Paketdatenbank in /var/db/pkg speichert nicht alle notwendigen Informationen, sie ist langsam, ineffizient und fehleranfällig.
- Die im Basissystem enthaltenen pkg_* Tools sind ebenso fehlerhaft und bieten nicht genug Funktionen.
In der Vergangenheit gab es mehrere Versuche um beide Nachteile herumzufrickeln. Reine Port-Tools wie Portmaster wurden erweitert, um auch Binärpakete verwalten zu können. Portupgrade implementierte seine eigene Datenbank. Dennoch erreicht man nie die Klasse, die Nutzer voin Linuxdistributionen gewohnt sind. Aus diesem Grund wurde in den letzten Monaten der Binärpaketmanager "pkgng" geschrieben. Die Designziele sind:
- Ersetze /var/db/pkg durch eine neue Datenbank, die schneller und robuster als die derzeitige Struktur ist. Die Wahl fiel auf eine sqlite-Datenbank.
- Erlaube weitere Metadaten und verändere das Paketformat so, dass es diese Metadaten speichern kann. Diese Metadaten werden nun in Form von YAML-Dateien gespeichert, denn dieses Format ist problemlos von Menschen lesbar und auch maschinell leicht zu handhaben.
- Entkoppele das Basissystem von den Ports. So wie ich es verstanden habe, soll pkgng nicht mehr Teil des Basisystems werden, viel mehr ein integraler Bestandteil der Ports. Es muss im Rahmen der Installation des Portstree installiert werden. Vielleicht kann hier noch jemand etwas zu sagen, der mit der Materie vertrauter ist.
- Stelle ein Interface bereit, was sich an gängigen Linux-Paketmanagern orientiert. pkgng ist daher ein Hybrid geworden, der Funktionen aus apt, yum, packman und einigen anderen Tools aus der Linuxwelt kombiniert.
Die unten stehende E-Mail erläutert pkgng noch ein wenig genauer und gibt Hinweise zu Benutzung des Tools. Es ist geplant, pkgng 1.0 gemeinsam mit FreeBSD 9.1 auszuliefern, was für den Sommer geplant ist. BITTE BEACHTET: Ist die Die Paketdatenbank muss konvertiert werden! Die E-Mail sagt nichts dazu, aber es scheint, dass der Weg zurück zu den bisherigen pkg_* Tools damit versperrt wird!
Code:
Hi,
pkgng has just reached the beta phase, and has now found its way to the
ports tree (disabled by default).
1/ Why pkgng?
------------
Our current pkg_install tools are showing their age, are hard to maintain,
and they lack features:
- missing metadata
- no upgrade support
- no repository support
- no fine dependency tracking
- no modern binary package management
- and many others
Having old tools makes it hard to improve the ports infrastructure, as a
result lots of hacks have found their way into the different Mk/bsd.*.mk
files to work around pkg_install limitations plus there are lots of hacks
in the packages metadata itself such as @comment which are not comments,
and so forth.
We have people writing tools to improve the situation (portmaster and
portupgrade to name two), but they are limited by and can become quite
complicated to maintain because of the pkg_install limitations.
2/ What it is?
--------------
It is a tool that is designed to replace pkg_install and provide modern
features to advance package management on FreeBSD.
It has been done with compatibility in mind. Most of the ports tree are
able to build on pkgng without modification (21500 successful packages is
the highest pkgng score so far). The missing ones will be easily fixed
with pkgng in ports.
It has been done with ease of migration in mind. It is easy to migrate
from pkg_install to pkgng. (Please note that going backwards is not
possible.)
It has been done with FreeBSD features in mind: it supports chroot, jails,
rcng, etc.
It has been done with scripting features in mind: 'pkg query' will allow
you to query almost everything from the pkgng database in a script friendly
way.
It has been done with improvement in mind: it doesn't require a privileged
account to create packages with root files in it; it is already able to
package from a stage/fakeroot/name_it_like_you_want directory; it is also
able to fake the package creation to directly install the package from that
fake/stage/whatever directory.
It has been done with human readability in mind: the new metadata is
stored in YAML format; the plist keywords can be extended with YAML (for
the ports).
It has been our thinking that the pkg binary is not able to please
everyone's needs, so it has been written on top of a library which can be
used by any other third party tools. (Think about packagekit, or ruby
binding for portupgrade for example, or any other usage like these).
pkgng is the result of my long studies and reflection about packaging
(studying what is done elsewhere: apt/dpkg, yum/rpm, pacman, aix, solaris,
netbsd, openbsd) and how to have something that tries to take the good
ideas from them, but tries not to take the *over engineered* complicated
parts. And most importantly, tries to do it the FreeBSD way: which means
it should work with the ports tree as-is (and help improve it in the future).
3/ Roadmap
----------
We plan a very long beta phase with lots of beta versions, released as
often as possible to ease testing and help improve the tool as much as
possible.
The goal, now that we are in beta is to not break anything for users, which
means that pkgng will be able to safely upgrade itself. (No real breakage
occurred during the alpha phase; expect even less in beta.)
Most of the big features are implemented, so now if you have a
revolutionary idea that breaks everything, it won't find its way into pkgng
1.0. You can still provide it for pkgng 2.0.
1.0 is not revolutionary because of the way that it is full of workarounds
to allow compatibility with the current ports tree. At some future time
(TBD), once we have dropped pkg_install support, things will be able to
move forward faster.
pkgng will live in the ports tree, so it will evolve with the
infrastructure, allowing us not to have to wait for the EOL of a release to
be able to move forward to new features.
The library API is currently not considered stable; it will be designated
stable as of pkgng 2.0. Therefore, if you are going to use the library in
a third party project, you can expect some breakage from time to time. Of
course, we will avoid breakage as much as possible.
The plan is to have pkgng 1.0 ready and rock solid for 10.0-RELEASE and
9.1-RELEASE.
The more testers/contributors we have, the faster we can go, and the faster
we go, the faster we can drop pkg_install and improve our port
infrastructure.
(Note: due to limitations in FreeBSD 7.x, we do not plan to backport
there.)
4/ pkgng itself
----------------
pkg add: add packages the old way (should be avoided by users)
pkg audit: audit the installed packages for vulnerabilities
pkg autoremove: interactively propose packages to be removed that were
installed automatically (as a dependency) and not depended on anymore
pkg check: check the installed packages database, prompting for
inconsistency and proposing to try to fix it
pkg clean: cleanup the package cache from binary installation (from
repositories)
pkg delete: deinstall packages
pkg info: query information from the installed packages in a user-friendly
way
pkg install: install packages from a remote repository
pkg query: query information from the installed packages in a
script-friendly way
pkg register: register packages in the database, synchronising the files
from a fake/stage directory, or with already installed files (like
currently with ports)
pkg search: search the remote database for packages
pkg update: update the remote repositories databases
pkg updating: scan the installed ports and show all UPDATING entries that
affect one of the installed ports (same as old pkg_updating)
pkg upgrade: perform a full binary upgrade of the installed packages
pkg version: determine whether package(s) need to be updated (same as old
pkg_version)
pkg which: determine which package owns a file
pkg2ng: convert a pkg_install installed database to a pkgng installed
database (you would need the ports tree for pkg2ng to be able to gather
missing information about packages)
Sample output of pkg info:
$ pkg info -f libreoffice:
Name : libreoffice
Version : 3.4.4
Origin : editors/libreoffice
Prefix : /usr/local
Categories : editors
Licenses : MPL & LGPL3
Maintainer : office@FreeBSD.org
WWW : http://www.libreoffice.org/
Comment : Full integrated office productivity suite
Options :
DEBUG: off
GNOME: off
GTK: on
JAVA: off
KDE4: off
MMEDIA: off
PYUNO: off
SDK: off
SYSTRAY: off
WEBDAV: off
Flat size : 319 MB
Description :
LibreOffice is the free power-packed Open Source personal productivity suite for
Windows, Macintosh and Linux, that gives you six feature-rich applications for
all your document production and data processing needs: Writer, Calc, Impress,
Draw, Math and Base.
WWW: http://www.libreoffice.org/
5/ what is missing
------------------
- for 1.0:
* currently the user handling is done using the @exec/@unexec scripts from
the ports; we need to finish the switch to the pw_/gr_util API to have
cleaner handling.
* better error reporting; lots of error/warning messages are currently
technical and need to be improved to become more user-friendly.
* better documentation; pkgng does a lot of things (more than what is
described here) and needs to be documented. We lack enough native English
speakers that are able to correctly document everything.
* bug hunting and fixing.
- for future:
* capsicum: during EuroBSDCon some ideas were shared on how we can
capsicumize pkgng, and more generally, increase security in package
management; we need to have more thought about this subject.
* improved arch support; currently pkgng is able to prevent installing
amd64 packages into an arm box for examples, but we can go further and
have real arch handling -- packaging noarch packages only once for all
architectures (shell scripts, data, etc) and share them between the
repository. This would reduce the size of the repositories and speed up
the packages building process.
* incrementally updated packages (diff packages): this is a really
complicated task but could be done, would need a clean design.
* abstract/alternative packages (e.g. "provide http_server")
* having a real sat solver for the dependency tree. Currently we have a
really simple and minimalistic solver which works well but if we can to go
to an even finer package management we would need a real solver.
* many more to join the project to share your ideas.
Keep in mind that in pkgng, every thing should be and will remain simple, so
when you come up with ideas, try providing a simple and clean design :)
to use pkgng:
echo WITH_PKGNG=yes >> /etc/make.conf
make -C /usr/ports/ports-mgmt/pkg install clean
Some links:
http://wiki.FreeBSD.org/pkgng
http://github.com/pkgng/pkgng
Note that on github you can find a patch for portmaster (against 3.10)
On behalf of the pkgng team
Bapt