External dependency solvers in APT

Back in April I talked about me being in Paris, I returned in May but kept more or less silent about it.

Nothing has really changed since then, I just want to write down now how somebody can play with other solvers now before it hits possibly a bigger audience (in case someone actually reads changelogs or such silly stuff).

All you need is:

  • APT in a 0.8.16~ version (debian-experimental has them, ubuntu-oneric too)
  • apt-cudf from debian-experimental
  • an external solver like aspcud from debian-unstable/testing

And you are ready to have some fun, try e.g.
apt-get install awesome --solver aspcud

You will notice: It’s relatively slow, compared to the internal solver. It might not behave like you would expect the internal solver to behave in regards to pinning or alike.

Best example is APT’s own external solver ‘apt’ – yes, you can use the internal solver as an external solver in case you find another program implementing EDSP – which doesn’t implement pinning and a few other bits so far. You are right than you think that this one is mostly only useful for debugging. There is another interesting one: ‘dump’ – imagine what it does…

So, why is this useful you might ask: In case you find a dependency problem you can now dump this problem easily and run different solvers on it. You can also work on developing the next-generation hyper-intelligent solver and be able to use it in day-to-day situations with the tools you usually use. And as a bug-hunter, you can if you have such a dump reproduce the problem more easily and get your hands dirty while fixing the bug. So this EDSP thing is kind of technical preview. It’s not planed to drop the internal solver from APT now or in the near future, don’t worry. It’s mostly here to enable researchers and developers to play with new deployments in real world situations so that at the end they create something which can be used for implementations which can really be used by “normal” end-users.

(and its here to enable me to throw the sentence: “Heh, if you thing the apt solver is so dumb and you could do it so much better: Why don’t you do it for the benefit of all of us?” at everyone complaining without hard facts just because they think nobody listens who will hunt them down and force them to solve all these “utterly trivial problems” on their own. Yes, I am looking at YOU ! ;) )

Posted in Debian | Tagged | 3 Comments

Return of the Shiva II

(this is just a fancy reminder for myself for the next time I trash my sheeva)

Upgrading the u-boot of the sheevaplug is super easy.

In the unlikely event that something goes wrong and you trashed the u-boot, so that the sheeva doesn’t respond to anything anymore, don’t panic! It’s not to hard to flash u-boot again without a working instance on the device itself.

First of all, install openocd on whatever you have left working (e.g. your laptop), ensure that the usb cable is still well-connected and open three shells, one root (#) and two users ($).

$ screen /dev/ttyUSB0 115200

# openocd -f /usr/share/openocd/scripts/board/sheevaplug.cfg

$ telnet localhost 4444
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger
> sheevaplug_init
target state: halted
target halted in ARM state due to debug-request, current mode: Supervisor
cpsr: 0x000000d3 pc: 0xffff0000
MMU: disabled, D-Cache: disabled, I-Cache: disabled
> nand probe 0
NAND flash device 'NAND 512MiB 3.3V 8-bit (Hynix)' found
> nand erase 0 0x0 0xa0000
erased blocks 0 to 5 on NAND flash device #0 'NAND 512MiB 3.3V 8-bit'
> nand write 0 /tmp/u-boot.kwb 0 oob_softecc_kw
> resume

(especially the ‘nand write’ needs a bit of time and produces no output, so calm down, that’s okay)

In the first shell with the tty connection we should see now our lovely u-boot again!

Watch out, for me on this try, u-boot claimed that my configuration was broken and used its default. Not a problem for me, i just redid what is described on the install page above:
(I would have needed to anyway, as ‘mmcinit’ changed to ‘mmc init’)
>> setenv bootargs_console console=ttyS0,115200
>> setenv bootcmd_mmc 'mmc init; ext2load mmc 0:1 0x00800000 /uImage; ext2load mmc 0:1 0x01100000 /uInitrd'
>> setenv bootcmd 'setenv bootargs $(bootargs_console); run bootcmd_mmc; bootm 0x00800000 0x01100000; reset'
>> saveenv

‘reset’ is added by me to restart the sheeva in case the bootcmd failed. ‘mmcinit’ used to fail sometimes with the old u-boot, now it seems to be better, but you can never be sure…

So, relatively easy: Welcome back sheeva!

Let’s see then I am able to trash it again…

Disclaimer: Yes, the upgrade description linked above is completely correct and should work flawless for everyone. Seems like I overlooked a ‘FAT entry invalid’ message or something like that so I flashed nothing to the place u-boot belongs to…

P.S.: New posts related to APT changes? Mhhh. Not too soon, but i will try…

Posted in Hardware | Tagged | Leave a comment

One week in Paris

(can’t help me, the title sounds like a good remake of bad stuff – I mean, the remake is bad, but double bad is good, isn’t it?)

I am back. After my sheeva and then my kindle returned back to me, I returned home, too. Last week was one of the craziest in my life: You are more or less alone in a big city, hundreds of people passing by and you understand nobody. The only language I can really speak is German, but the masochist in me accepts invitations to America where my rusty English allows me to survive, but how he could accept an invitation to Paris in the knowledge that I don’t speak a single word of French?!?
I learned two words: ‘Bonjour’ (hello) and ‘Sortie’ (exit) in this week. As you can see, I am not a fast learner…

Anyway, it’s properly more interesting what I did in Paris: Stefano happens to be working for one of the university of Paris and in this function for IRILL on mancoosi. In a nutshell: Defining and organizing dependency solving battles by defining a protocol all solvers have to understand and what the response should be.

The week was therefore dedicated to working out a way for APT to talk to external solvers, so that we can use the battle survivors as external solvers for (maybe) better solutions. I will write a bit more detailed how this works (or not) after finishing it in a second stay in may, but so far we have an EDSP which defines an intermediate scenario format which is near to CUDF, but doesn’t rewrite versionnumbers and alike as this is internal business a solver doesn’t need to do to work with APT. So in practice the cudf solvers will most likely use a common preprocessor on this scenario and be happy. A common postprocessor will then reformat the cudf response to a diff style answer which will be passed to APT which reacts on it and does his “magic”.

EDSP is currently an early draft and the implementation in APT more or less half-way finished while EDSP<->CUDF translator is promising, too. All in all we will most like see something soon in may modulo that it will need an abi-break in APT which needs a lot of time to be allowed in unstable most of the time.

Time will tell, so far at least from the water-level observation one-man-team here. ;)

Posted in Debian | Tagged | 1 Comment

Return of the kindle

Amazons Kindle is a nice device. And can’t say like others that my reading habit exploded, but as i am traveling a lot by train between hometown and university its quiet useful to read a bit of university stuff on it, play a silly game or really reading a book. All in all it well worth the investment last year in Orlando, FL. Shipping and Taxes always kill the fun in buying something online if it’s not sold in Germany…

Anyway, a week before Paris I had a problem: I planed reading “Ender’s Game” as I thought the reservation-only train will be overcrowded so no space to “play” on my laptop – but my kindle only showed my screensaver and didn’t reacted to anything. I noticed a one inch thick line in the middle of the device all across the screen and a few others just a few dots thick and high. What the hell? First my sheeva was fried and now a broken screen on my kindle.

Thankfully amazon saved my travel. After a lovely phone call (= none of “please hold the line”, silly music, “have you tried”-questions after describing I already did that) the women on the other side of world and phoneline said “we will sent a replacement to …… I can’t pronounce that!” :D
Germany streets end most of the time with “straße” (which is our word for street). Please try to pronounce it now if you didn’t even know the second last letter. :P

It was sorted out and just a few days later well in time for Paris UPS handed me the package over. Incredible
service! I am saying that now before even returning the broken one, so the process is not really over, but I am so fascinated that I had to say that now as I would be more than shocked now if something would go mad now…

Oh and for the record: I don’t know why the screen broke in the first place: I had it lying around openly so in theory nothing which could have caused it – beside our cat maybe. She likes sitting on top of basically everything… but I have no prove, I am just trying to not give here an opportunity again. Tiger, i am watching you! ;)

(this post is backdated to fit better into the chronology)

Posted in Hardware | Leave a comment

Return of the Shiva

Wikipedia says that shiva is the hindu deity known as the “Destroyer”. Interesting that GlobalScale choose “sheeva” as the name for their first plug generation. Sounds a bit like a self-fulfilling prophecy…
Exploded power supply unit of my sheeva
The internet is full of stories and images like the one on the left – “exploded” power supply units cause by… nobody knows for sure. It seems to be that attaching passive usb devices is not a good idea. My harddrive was self-powered but I had also a dvb-t card attached on a non-active usb hub… bad.

As a proper replacement PSU shipped by NewIT would be available only in late February I crafted a replacement together by myself from an unused ac/dc adapter as an external powersupply. I am always a bit scared by hardware-modding but stripping and connecting the wires was easy after finding descriptions like this so my sheeva raised from the death – lets see how long this will last. Next week I will properly shop for an active hub to decrease the PSU risk further. At some point I should also get a real replacement PSU, but at least I can wait and search now for one where the cost isn’t doubled (or even more) by shipping costs…
(Why is there no German based distributor, who has one available?)

Posted in Hardware | Tagged | Leave a comment

Oh my god! They are coming!

Yesterday, Michael opened the flood gates and today I finally published what was in my private basement for a while now. So (Keep your kids and servers at home and offline) they are coming – all the cool, useful and nifty stuff included in my patches! ;)

Backports and experimental-user will hopefully find the ReleaseCandidate switching useful. It’s a behavior change in APT, but as the previous behavior was a failure I assume nothing will break to badly now – and even if it does, we have a full release circle to fix it. So what does it do?
Lets look at an example:
apt-get install grub-pc/experimental -s
Reading package lists... Done
Building dependency tree
Reading state information... Done
Selected version '1.99~rc1-1' (Debian:experimental [i386]) for 'grub-pc'
Selected version '1.99~rc1-1' (Debian:experimental [i386]) for 'grub-common' because of 'grub-pc'
The following extra packages will be installed:
grub-common
Suggested packages:
multiboot-doc grub-emu xorriso desktop-base
The following packages will be upgraded:
grub-common grub-pc
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

So as you can see, apt-get switches automatically the candidate for grub-common to experimental, too, as the grub-pc from experimental demands it (with a versioned dependency). Quite cool as the usual “-t experimental” would cause also dependencies which would be satisfiable by the usual default release to be upgraded to experimental.

And as we are talking about experimental: I experimented with symbol files for libapt and my branch includes some. Far from perfect and they will need a lot of love to be really useful (as a stepstone to a “real” library) but they at least doesn’t make the situation worse and solved even a (more) theoretical partial upgrade problem.

Of more practical use for now is the stuff I changed in the acquire system in preparation for bigger changes we will need to finally be able to cleanly download addition metadata. APT uses the clearsigned InRelease file if possible instead of Release and its detached Release.gpg and description translations doesn’t need to be whitelisted any longer as APT now downloads the i18n/Index file to see if the translation is available at all.

More cool acquire stuff will properly need an ABI break for so silly reasons as a new virtual method or alike (I tricked a bit so the current stuff doesn’t need it). Michael started to enhance this situation by preparing for introducing dpointers (another step to a library) beside introducing ‘apt-get download’ and ‘apt-get changelog’! One step closer to world domination. :D

If that wouldn’t be cool enough by itself now two weeks ago I was at the “Cross-Distro App Installer Meeting” which has the codename “AppStream” now. Very interesting stuff not only because of the metadata someone need to write and download, but also I meet Michael in person (okay, that’s kind of normal) but also a long-time debianer with Enrico, who is the “daddy” of debtags and apt-xapian-index. He was very close to convenience me to apply for D{D,M}, but I managed to withstand… The punishment followed closely in form of apt 0.8.10.2 and 0.8.10.3 and the attached discussion to get them approved by the release team – just two weeks before release. I was near to the point of posting a ‘How to delay a debian release as a total stranger’. As a D{D,M} the ‘he is a nobody’-bonus would have been gone… Which could have been an the last needed bit as my help in smoothing the upgrades of KDE was obviously not enough… ;)

Anyway, another interesting point of these three days was noticing zypper and its underlying sat-solver. So the RPM world has something usable already while we “hope” for mancoosi more or less. In general, listening and talking to the “other side” was pretty interesting. I can’t imagine 10 fanboys sitting around a table talking civilized, but developers seems to be really good at it :P The RPM-World was quite open to what debian has to offer and we looked with big eyes at what they have. It’s strange that you are basically working on the same stuff, with nearly the same environment but without knowing a single bit about the other because they use a different package manager – dpkg vs. rpm — there are ports of APT available for rpm to enlight the poor souls. ;)

Cross-distribution seems to become (or will hopefully be) the next big thing…

Disclaimer: All new stuff described here is currently only in my branch, but will migrate to the official branch after (hopefully) passing review and therefore be part of an upcoming release of APT (properly 0.8.11) for wheezy. Or to word it differently: This is and will be not available in squeeze!

Posted in Debian | Leave a comment

Merry package management days!

There are times that everyone seems to be interested in package management and usually we call this time “Christmas”. A single person gets the job of sending every child in the world more or less at the same time one or more presents – that sounds completely impossible. You know, there are even pseudo scientific studies that Chris Kringle would melt instantly in the try to travel fast enough… Yet alone, the production of all these presents at a single place hidden from the rest of the world. But is it really unrealistic? Linux has a marketshare under 1% and is therefore perfectly hidden for the majority of people. Debian provides more than 30.000 packages – more than any single person will ever need and such a number is by far not easy.
Many people are working on this, still, there is a single team responsible for the final shipment: The release team, unbelievable that this works out… the release team has only a single advantage: Debian has no predefined release day – debian releases then ready. :)

Anyway i am losing track, i want to say something completely different: I am personally not a big fan of this kind of “package management”. I mean, why do we need special days to exchange gifts? Don’t get me wrong, i love these days with the family and friends, but i am very happy that my family erased this strange requirement to get something for everyone as this generates only stress and even a whole new market-segment after Christmas which is happily served by eBay and co… (given that they even show ads regarding this each year after Christmas).

So, in conclusion, have fun with your family!
(And be sure to trust your package management, you know that there is only one ;))

P.S.: These days are the cause for my inactivity in any serious APT business but i will (hopefully) be back 2011, just in case you wonder…

Posted in Debian | Leave a comment