Go to the content
or

Debian

Full screen Suggest an article
 RSS feed

Planet Debian

July 1, 2016 14:34 , by valessiobrito - | No one following this article yet.

Planet.Debian is a website that aggregates the blogs of many Debian contributors. Planet maintainers can be reached at planet at debian.org


Sean Whitton: DebCamp/DebConf17: reports on sprints and BoFs

August 17, 2017 22:21, by Planet Debian - 0no comments yet

In addition to my personal reflections on DebCamp/DebConf17, here is a brief summary of the activities that I had a hand in co-ordinating.

I won’t discuss here many other small items of work and valuable conversations that I had during the two weeks; hopefully the fruits of these will show themselves in my uploads to the archive over the next year.

Debian Policy sprint & BoF

  • released version 4.0.1.0 of the Policy Manual

  • figured out documenting reproducibility in policy. Formulating the wording turned out to be easier than I had expected

  • approx. ten years after they were first published, incorporated marga’s maintscript flowcharts into policy proper

  • converted policy from docbook to rST built with the Sphinx toolchain. Many, many thanks to Hideki Yamane and David Bremner for helping Russ and I get this merged to our master branch

  • triage of every single bug against policy, and mass closure of inactive bugs, bringing the total down from more than 200 to around 125

  • conversations with Technical Committee members about how the two teams can help each other’s work (mainly us helping them to help us!)

  • conversations about how we handle disagreement and plans to streamline our overly complex BTS usertags (watch this space)

  • very useful input from policy consumers about how the upgrading checklist is formatted, and how we can recruit more people to get patches written

Debian Emacs Team meeting/sprint

  • plans to finally drop our emacsXY binary packages, and just have a single version of Emacs in the archive, so that we no longer have to deal with bugs due to someone still having emacs21 installed (David’s idea; Rob’s implementation; Sean’s mostly-helpful comments)

  • other plans to simplify and otherwise improve the Debian Emacsen policy

  • finally finished off the work needed to RM emacs24—nine months later—including a lot of NMUs

  • mentoring a junior team member

Unfortunately we didn’t make any significant progress towards converting all addons to use dh_elpa, as the work is not that much fun. Might be worth a more focused sprint next year.

Git for Debian packaging BoF & follow-up conversations

The BoF was far more about dgit than I had wanted; however, I think that this was mostly because people had questions about dgit, rather than any unintended lecturing by me.

I believe that several people came away from DebConf thinking that starting to use dgit would improve Debian for themselves and for users of their packages.



Sean Whitton: I went all the way to Montréal for DebConf17, and all I got was a new MUA

August 17, 2017 21:50, by Planet Debian - 0no comments yet This year’s group photo (by Aigars Mahinovs). I really like the tagline

On Sunday night I got back from Montréal, where I attended both DebCamp17 and DebConf17. It was a wonderful two weeks. All I really did was work on Debian for roughly eight hours per day, interspersed with getting to know both people I’ve been working with since I first began contributing to Debian in late 2015, and people I didn’t yet know. But this was all I really needed to be doing. There was no need to engage in distracting myself.

I enjoyed the first week more. There were sufficiently few people present that you could know at least all of their faces, and interesting-sounding talks didn’t interrupt making progress on one’s own work or unblocking other people’s work. In the second week it was great to meet people who were only present for the second week, but it felt more like regular Debian, in that I was often waiting on other people or they were waiting on me.

While I spent one morning actually writing fresh code, and I did a fair amount of pure packaging work, the majority of my time was poured into (i) Debian Policy; (ii) discussions within the Emacs team; and (iii) discussions about dgit. This was as I expected. During DebConf, it’s not that useful to seclude oneself and sufficiently reacquaint oneself with a codebase that one can start producing patches, because that can be done anywhere in the world, without everyone else around. It’s far more useful to bring different people together to get projects unblocked. I did some of that for my own work, and also tried to help other people’s, including those who weren’t able to attend the conference.

In my ordinary life, taking a step back from the methods by which I protect my PGP keys and other personal data, I can appear to myself as a paranoid extremist, or some kind of data hoarder. It was comforting to find at DebConf plenty of people who go way further than me: multiple user accounts on their laptop, with separate X servers, for tasks of different security levels; PGP keys on smartcards; refusal to sign my PGP key based on government-issued ID alone; use of Qubes OS. One thing that did surprise me was to find myself in a minority for using the GNOME desktop; I had previously assumed that most people deep in Debian development didn’t bother with tiling window managers. Turns out they are enthusiastic to talk about the trade-offs between window managers while riding the subway train back to our accommodation at midnight—who knew such people existed? I was pleased to find them. One evening, I received a tag-teamed live tutorial in using i3’s core keybindings, and the next morning GNOME seemed deeply inelegant. The insinuation began, but I was immediately embroiled in inner struggle over the fact that i3 is a very popular tiling window manager, so it wouldn’t be very cool if I were to start using it. This difficulty was compounded when I learned that the Haskell team lead still uses xmonad. The struggle continues.

I hope that I’ve been influenced by the highly non-judgemental and tolerant attitudes of the attendees of the conference. While most people at the conference were pretty ordinary—aside from wanting to talk about the details of Debian packaging and processes!—there were several people who rather visibly rejected social norms about how to present themselves. Around these people there was nothing of the usual tension. Further, in contrast with my environment as a graduate student, everyone was extremely relaxed about how everyone was spending their time. People drinking beer in the evenings were sitting at tables where other people were continuing to silently work on Debian. It is nice to have my experience in Montréal as a reference to check my own judgemental tendencies.

I came away with a lot more than a new MUA: a certainty that I want to try to get to next year’s conference; friends; a real life community behind what was hitherto mostly a hobby; a long list of tasks and the belief that I can accomplish them; a list of PGP fingerprints to sign; a new perspective on the arguments that occur on Debian mailing lists; an awareness of the risk of unconsciously manipulating other community members into getting work done.

With regard to the MUA, I should say that I did not waste a lot of DebConf time messing with its configuration. I had actually worked out a notmuch configuration some months ago, but couldn’t use it because I couldn’t figure out how to incorporate my old mail archives into its index. Fortunately notmuch’s maintainer is also on the Emacs team … he was able to confirm that the crazy solution I’d come up with was not likely to break notmuch’s operating assumptions, and so I was able to spend about half an hour copying and pasting the configuration and scripts I’d previously developed into my homedir, and then start using notmuch for the remainder of the conference. The main reason for wanting to use notmuch was to handle Debian mailing list volume more effectively than I’m able to with mutt, so I was very happy to have the opportunity to pester David with newbie questions.



Reproducible builds folks: Reproducible Builds: Weekly report #120

August 17, 2017 21:48, by Planet Debian - 0no comments yet

Here's what happened in the Reproducible Builds effort between Sunday 6th and Saturday 12th August 2017:

Notes about reviews of unreproducible packages

13 package reviews have been added, 7 have been updated and 34 have been removed in this week, adding to our knowledge about identified issues.

Packages reviewed and fixed, and reproducibility related bugs filed

Upstream packages:

Other bugs filed

  • During our reproducibility testing, Adrian Bunk filed 48 FTBFS bugs this week.

diffoscope development

trydiffoscope development

tests.reproducible-builds.org

  • Mattia:
    • Notify the#debian-reproducible-changes` IRC channel for unreproducible -> FTBFS transitions.
    • Update squid.conf for all nodes to 5.2.23 (and fixup some).
    • Enable the Munin Squid plugin on the Codethink arm64 nodes as well.
    • Force reconfiguration of Apache and Munin when update_jdn.sh is updated.
  • Holger worked on slides for his DebConf17 BoF about migrating to jenkins.debian.org, which affects tests.r-b.o as well.

Misc.

This week's edition was written by Chris Lamb & Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.



Julian Andres Klode: Why TUF does not shine (for APT repositories)

August 17, 2017 19:49, by Planet Debian - 0no comments yet

In DebConf17 there was a talk about The Update Framework, short TUF. TUF claims to be a plug-in solution to software updates, but while it has the same practical level of security as apt, it also has the same shortcomings, including no way to effectively revoke keys.

TUF divides signing responsibilities into roles: A root role, a targets rule (signing stuff to download), a snapshots rule (signing meta data), and a time stamp rule (signing a time stamp file). There also is a mirror role for signing a list of mirrors, but we can ignore that for now. It strongly recommends that all keys except for timestamp and mirrors are kept offline, which is not applicable for APT repositories – Ubuntu updates the repository every 30 minutes, imagine doing that with offline keys. An insane proposal.

In APT repositories, we effectively only have a snapshots rule – the only thing we sign are Release files, and trust is then chained down by hashes (Release files hashes Packages index files, and they have hashes of individual packages). The keys used to sign repositories are online keys, after all, all the metadata files change every 30 minutes (Ubuntu) or 6 hours (Debian) – it’s impossible to sign them by hand. The timestamp role is replaced by a field in the Release file specifying until when the Release file is considered valid.

Let’s check the attacks TUF protects again:

  • Arbitrary installation attacks. – We protect against that with the outer signature and hashes
  • Endless data attacks. – Yes, we impose a limit on Release files (the sizes of other files are specified in there and this file is signed)
  • Extraneous dependencies attacks – That’s verified by the signed hashes of Packages files
  • Fast-forward attacks – same
  • Indefinite freeze attacks – APT has a Valid-Until field that can be used to specify a maximum life time of a release file
  • Malicious mirrors preventing updates. – Well, the user configures the mirror, so usually not applicable. if the user has multiple mirrors, APT deals with that fine
  • Mix-and-match attacks – Again, signed Release file and hashes of other files
  • Rollback attacks – We do not allow Date fields in Release files to go backwards
  • Slow retrieval attacks – TUF cannot protect against that either. APT has very high timeouts, and there is no reasonable answer to that.
  • Vulnerability to key compromises – For our purposes where we need all repository signing keys to be online, as we need to sign new releases and metadata fairly often, it does not make it less vulnerable to require a threshold of keys (APT allows repositories to specify concrete key ids they may be signed with though, that has the same effect)
  • Wrong software installation. – Does not happen, the .deb files are hashed in the Packages files which are signed by the release file

As we can see, APT addresses all attacks TUF addresses.

But both do not handle key revocation. So, if a key & mirror gets compromised (or just key and the mirror is MITMed), we cannot inform the user that the key has been compromised and block updates from the compromised repository.

I just wrote up a proposal to allow APT to query for revoked keys from a different host with a key revocation list (KRL) file that is signed by different keys than the repository. This would solve the problem of key revocation easily – even if the repository host is MITMed or compromised, we can still revoke the keys signing the repository from a different location.


Filed under: Debian, Ubuntu

Shirish Agarwal: Composers are not given due recognition

August 17, 2017 18:17, by Planet Debian - 0no comments yet

Beware – some youtube-links would be shared in this entry, sorry couldn’t find a better/easier media platform to work with. If anyone knows any other platform or wants to suggest, feel free to either mail me or let me know in comments.

I want to start today’s sharing with a picture of Ganesha I saw today. It is and was public art hence sharing it without an issue.

Sketch of Ganesha/Ganapati

This is starting of festivities time in India and Ganesha or Ganpati is looked up as a good omen in India.

The festival of Ganesh Chaturthi would be starting on the 25th of August and is a sight to behold. Just like Rio has its carnival, Ganesh Chaturthi is also a carnival. We also have parades where people come with Pandals (or temporary structures)

The mythology says he has a sweet tooth (hence lot of distribution of sweets, especially modak) and anything which might be troubling people, he creates solutions for them.

Here is one video of how people celebrate his immersion in India. This is from my home-town few years ago, every year the madness and the celebrations are becoming more and more. People from far off come to see how we celebrate and see how different people make their Pandals. While some are with music, others are with social messages. Usually people start going to see these structures after dusk and return home way after midnight or early morning. I hope to do this endeavour after many years. One is drunk from hearing all sorts of different kids of music, decoration, messages, a feast and a strain to all the senses.

If one is interested one can find more info. at https://en.wikipedia.org/wiki/Ganesh_Chaturthi

After quite a bit of time, I wrote an article about various foss internships which I knew besides GSOC over the years. I finally penned them down at https://itsfoss.com/best-open-source-internships/

Interestingly, I was amazed to see that all FOSS U.S. projects (outside of GSOC) are for students who are either living or studying in U.S. and have a student work visa (which from private discussions I came to know is lot harder to get nowadays than before). Except for the National Science Foundation (NSF) which probably has U.S. defence relations and hence they might be sensitive, I fail to understand other institutes preferences for only getting people from the U.S. Dunno if this is due to the present President Trump or these policies had been there before. It would be nice and interesting if people in the know can share.

What has also been interesting to watch is Mr. Trump blaming low-cost manufacturing centres like India and China when as far as I recall, lot of manufacturing, specifically auto-mobiles manufacturing was shifted out of the U.S. to Ireland and other places years before which are relatively high-cost places (at least compared to India). I *believe* the change was as early as in 1980’s itself where India was insulated and had a limited market for everything (similar to Russian communism as shown in popular media but not so bad.)

Interestingly, it took almost a month for the perl 2.56 to make the transition smoothly. It took quite a bit of time for all the components to work together and be installable.

Also saw this few days back http://fortune.com/2017/04/12/auto-industry-decline/

While Tesla is expensive even by American standards the idea of lesser parts, lesser complexity and hence lower costs to use, maintain is good. I do hope that he and his team or any of the competitors do overcome the significant challenges. Any significant improvement in battery technology is bound to have huge impact in almost everything that is used in 21st century.

Two recent articles tell me the future may become present very quickly.

https://www.purdue.edu/newsroom/releases/2017/Q2/instantly-rechargeable-battery-could-change-the-future-of-electric-and-hybrid-automobiles.html

Toyota could finally start mass producing electric cars thanks to China

I do hope to see EV being prevalent before the next decade is over otherwise we don’t have any hope.

As for my health, I am much better than before. Just to share some stats, before my “illness” for lack of better word, I was 120 kgs. , when I was kept in the hospital for about 2-2.5 weeks I came down to 95 kgs. and now back upto 108 kgs. Do go for exercising every other day and trying to get back the strength, stamina and increasing a bit of both. Doctors have given me another 4-5 months after which a brain scan will reveal if there are any remaining blood clots in the brain or not.

Lastly, while it has become somewhat of a sensitive issue to love Muslims or to talk about their work in any field in the current political climate, there are 4-5 music pieces I listen whenever I can, especially before going to bed. While almost all the pieces have been sung and written by Muslims, sadly I don’t know who the composers of these beautiful songs are. While it is much easier to get the names of the singer and the lyricist, one of the more important roles in my view is the composer or/and music arranger. Without them, the songs would not have the same haunting quality that the songs have. While I have been lucky to find the names of the composer/music arranger but this is not the case if and when the songs comes on television. I do remember in old times in Radio they used to mention about who has given the music as well, dunno in modern times.

I am sharing the songs, and hopefully will also share the translations if I find on the web, please see the lyrics. The numbering is for convenience only and am torn in these 4-5 songs which is the best. Just to share these are all sufi love songs except the last one which I will share.

1.

Just lyrics – https://www.youtube.com/watch?v=ehqN6oTpmb8

Translation with video of song – http://www.bollynook.com/en/lyrics/6443/aaj-din-chadheya/

While there probably are stories with each song, I was lucky to find the story about this one. The lyrics of the song are actually a love lost Punjabi poet who writes in the memory of his beloved to which he could not marry and he pens those when standing in line for his liquor. The story goes on that he marries a girl later in life who bears a resemblance to his beloved whom he couldn’t forget till his dying day.

2.

The same song has been sung by different people and I love them all the more for it.

Translation – http://www.bollynook.com/en/lyrics/10703/o-re-piya/

3.

The translation – http://www.filmyquotes.com/songs/885

The translation of the song is a bit crude but then translations are supposed to be crude 😦

Anyways, the above song is what would be called a perfect Sufi song. I hope people enjoy the longing and the silence which follows this piece.

Another classic one

4.

English translation – http://www.ardhamy.com/song/aye-dil-e-naadan

While the song is from the movie Razia Sultana and was a flop as the movie was about Race and controversial then as it probably would be now. As seen in the other songs of the same genre, it has strands of longing, loneliness as seen of the above.

5. https://www.youtube.com/watch?v=tv242qOnHJA

This one is not a sufi song but I love all the women and the girls and the way they enhanced the song. I dunno how much they must have practised as it’s a very fast and peppy song and doesn’t give time to the singer to breathe except for that one section which has a bit of Carnatic music.

At the very end I would like to share http://www.globalrhythm.net/ I have found some interesting sounds on the site. Hope the site enriches you as well. FWIW I have no links with the site except as somebody who likes to diversify his music listening.

Lastly, for a long period of time, I had been hearing the criticism, especially for FOSS games that they don’t have AAA quality assets. Recently I came across a game called Starship Theory (sadly its only for MS-Windows)

You look at the game and see the number of videos the guy has made. What FOSS game developers can learn from this, you don’t need high-end 3-d bodies but only depth which can make FOSS games
earn a pretty bundle. I hope all upstream FOSS game developers understand and take

Hope you have a good time with all the ideas, anecdotes and videos I shared above.


Filed under: Miscellenous Tagged: #FOSS Internships, #Ganesh Chaturthi, #Ganeshji, #planet-debian, #Sufi Bollywood Music, FOSS, politics

Bits from Debian: Work on Debian for mobile devices continues

August 17, 2017 13:40, by Planet Debian - 0no comments yet

Work on Debian for mobile devices, i.e. telephones, tablets, and handheld computers, continues. During the recent DebConf17 in Montréal, Canada, more than 50 people had a meeting to reconsider opportunities and challenges for Debian on mobile devices.

A number of devices were shown at DebConf:

  • PocketCHIP: A very small handheld computer with keyboard, Wi-Fi, USB, and Bluetooth, running Debian 8 (Jessie) or 9 (Stretch).
  • Pyra: A modular handheld computer with a touchscreen, gaming controls, Wi-Fi, keyboard, multiple USB ports and SD card slots, and an optional modem for either Europe or the USA. It will come preinstalled with Debian.
  • Samsung Galaxy S Relay 4G: An Android smartphone featuring a physical keyboard, which can already run portions of Debian userspace on the Android kernel. Kernel upstreaming is on the way.
  • ZeroPhone: An open-source smartphone based on Raspberry Pi Zero, with a small screen, classic telephone keypad and hardware switches for telephony, Wi-Fi, and the microphone. It is running Debian-based Raspbian OS.

photo of Samsung, Pyra, N900, ZeroPhone, GnuK, and PocketCHIP - click to enlarge

The photo (click to enlarge) shows all four devices, together with a Nokia N900, which was the first Linux-based smartphone by Nokia, running Debian-based Maemo and a completely unrelated Gnuk cryptographic token, which just sneaked into the setting.

If you like to participate, please



Holger Levsen: 20170816-irssi-timezones

August 16, 2017 21:02, by Planet Debian - 0no comments yet

How to change irssi's timezone without restart

Happy birthday to all you lovely Debian people!

For my future self:

<Rhonda> | h01ger: /script exec $ENV{TZ} = 'Europe/Vienna';


Ross Gammon: My Debian &amp; Ubuntu work from April to mid-August 2017

August 16, 2017 17:16, by Planet Debian - 0no comments yet

Okay, so I have been slack with my blogging again. I have been travelling around Europe with work quite a bit, had a short holiday over Easter in Denmark, and also had 3 weeks of Summer Holiday in Germany.

Debian

  • Tidied up the packaging and tried building the latest version of libdrumstick, but tests had been added to the package by upstream which were failing. I still need to get back and investigate that.
  • Updated node-seq (targeted at experimental due to the Debian Stretch release freeze) and asked for sponsorship (as I did not have DM rights for it yet).
  • Uploaded the latest version of abcmidi (also to experimental), and again.
  • Updated node-tmp to the latest version and uploaded to experimental.
  • Worked some more on bluebird RFP, but getting errors when running tests. I still haven’t gone back to investigate that.
  • Updated node-coffeeify to the latest version and uploaded to experimental.
  • Uploaded the latest version of node-os-tmpdir (also to experimental).
  • Uploaded the latest version of node-concat-stream (also to experimental).
  • After encouragement from several Debian Developers, I applied to become a full Debian Developer. Over the summer months I worked with Santiago as my Application Manager and answered questions about working in the Debian Project.
  • A web vulnerability was identified in node-concat-stream, so I prepared a fix to the version in unstable, uploaded it to unstable, and submitted a unblock request bug so that it would be fixed in the coming Debian Stretch release.
  • Debian 10 (Stretch) was released! Yay!
  • Moved abcmidi from experimental to unstable, adding an autopkgtest at the same time.
  • Moved node-concat-stream from experimental to unstable. During the process I had to take care of the intermediate upload to stretch (on a separate branch) because of the freeze.
  • Moved node-tmp to unstable from experimental.
  • Moved node-os-tmpdir from experimental to unstable.
  • Filed a removal bug for creepy, which seems to be unmaintained upstream these days. Sent my unfinished Qt4 to Qt5 porting patches upstream just in case!
  • Uploaded node-object-inspect to experimental to check the reverse dependencies, then moved it to unstable. Then a new upstream version came out which is now in experimental waiting for a retest of reverse dependencies.
  • Uploaded the latest version of gramps (4.2.6).
  • Uploaded a new version of node-cross-spawn to experimental.
  • Discovered that I had successfully completed the DD application process and I was now a Debian Developer. I celebrated by uploading the Debian Multimedia Blends package to the NEW queue, which I was not able to do before!
  • Tweaked and uploaded the node-seq package (with an RC fix) which had been sitting there because I did not have DM rights to the package. It is not an important package anyhow, as it is just one of the many dependencies that need to be packaged for Browserify.
  • Packaged and uploaded the latest node-isarray directly to unstable, as the changes seemed harmless.
  • Prepared and uploaded the latest node-js-yaml to experimental.
  • Did an update to the Node packaging Manual now that we are allowed to use “node” as the executable in Debian instead of “nodejs” which caused us to do a lot of patching in the past to get node packages working in Debian.

Ubuntu

  • Did a freeze exception bug for ubuntustudio-controls, but we did not manage to get it sponsored before the Ubuntu Studio Zesty 17.04 release.
  • Investigated why Ardour was not migrating from zesty-proposed, but I couldn’t be sure of what was holding it up. After getting some help from the Developer’s mailing list, I prepared “no change rebuild” of pd-aubio which was sponsored by Steve Langasek after a little tweak. This did the trick.
  • Wrote to the Ubuntu Studio list asking for support for testing the Ubuntu Studio Zesty release, as I would be on holiday in the lead up to the release. When I got back, I found the release had gone smoothly. Thanks team!
  • Worked on some blueprints for the next Ubuntu Studio Artful release.
  • As Set no longer has enough spare time to work on Ubuntu Studio, we had a meeting on IRC to decide what to do. We decided that we should set up a Council like Xubuntu have. I drafted an announcement, but we still have not gone live with it yet. Maybe someone will have read this far and give us a push (or help). 🙂
  • Did a quick test of Len’s ubuntustudio-controls re-write (at least the GUI bits). We better get a move on if we want this to be part of Artful!
  • Tested ISO for Ubuntu Studio Xenial 16.04.3 point release, and updated the release notes.
  • Started working on a merge of Qjackctl using git-ubuntu for the first time. Had some issues getting going, so I asked the authors for some advice.



Bits from Debian: Debian turns 24!

August 16, 2017 15:50, by Planet Debian - 0no comments yet

Today is Debian's 24th anniversary. If you are close to any of the cities celebrating Debian Day 2017, you're very welcome to join the party!

If not, there's still time for you to organize a little celebration or contribution to Debian. For example, spread the word about Debian Day with this nice piece of artwork created by Debian Developer Daniel Lenharo de Souza and Valessio Brito, taking inspiration from the desktop themes Lines and softWaves by Juliette Belin:

Debian 24

If you also like graphics design, or design in general, have a look at https://wiki.debian.org/Design and join the team! Or you can visit the general list of Debian Teams for many other opportunities to participate in Debian development.

Thanks to everybody who has contributed to develop our beloved operating system in these 24 years, and happy birthday Debian!



Lisandro Damián Nicanor Pérez Meyer: Qt 4 removal in Debian testing (Buster)/unstable

August 15, 2017 16:50, by Planet Debian - 0no comments yet

We have been announcing it: we are going to remove Qt 4 during the Buster cycle.

Or at least that's the best outcome we can expect. Removing a very highly used library is hard, as Qt4's Webkit has proved. Qt 4 is long dead upstream and we have already started to need to patch it with untested patches as in the OpenSSL 1.1 case (will be in experimental in a few hours after this post).

We will try to put as less effort as possible in keeping it alive meaning that from now on if we need to patch it to make it support a newer lib or alike we will simply remove its support if possible. Using the OpenSSL case as an example, if we need to support any version > 1.1 we will simply remove the SSL support. That means things will break.

So, if you depend on FLOSS which is still based on Qt 4 be sure to try to port it. If you depend on a proprietary vendor software which uses Qt 4 then you better start telling them it's really time to update it. Really.

We will soon start filing bugs against packages using Qt 4. I'll update this blog post later to add that info.

For the Qt/KDE team, Lisandro.



Dirk Eddelbuettel: #9: Compacting your Shared Libraries

August 15, 2017 1:49, by Planet Debian - 0no comments yet

Welcome to the nineth post in the recognisably rancid R randomness series, or R4 for short. Following on the heels of last week's post, we aim to look into the shared libraries created by R.

We love the R build process. It is robust, cross-platform, reliable and rather predicatable. It. Just. Works.

One minor issue, though, which has come up once or twice in the past is the (in)ability to fully control all compilation options. R will always recall CFLAGS, CXXFLAGS, ... etc as used when it was compiled. Which often entails the -g flag for debugging which can seriously inflate the size of the generated object code. And once stored in ${RHOME}/etc/Makeconf we cannot on the fly override these values.

But there is always a way. Sometimes even two.

The first is local and can be used via the (personal) ~/.R/Makevars file (about which I will have to say more in another post). But something I have been using quite a bite lately uses the flags for the shared library linker. Given that we can have different code flavours and compilation choices---between C, Fortran and the different C++ standards---one can end up with a few lines. I currently use this which uses -Wl, to pass an the -S (or --strip-debug) option to the linker (and also reiterates the desire for a shared library, presumably superfluous):

SHLIB_CXXLDFLAGS = -Wl,-S -shared
SHLIB_CXX11LDFLAGS = -Wl,-S -shared
SHLIB_CXX14LDFLAGS = -Wl,-S -shared
SHLIB_FCLDFLAGS = -Wl,-S -shared
SHLIB_LDFLAGS = -Wl,-S -shared

Let's consider an example: my most recently uploaded package RProtoBuf. Built under a standard 64-bit Linux setup (Ubuntu 17.04, g++ 6.3) and not using the above, we end up with library containing 12 megabytes (!!) of object code:

edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so
-rwxr-xr-x 1 edd edd 12M Aug 14 20:22 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ 

However, if we use the flags shown above in .R/Makevars, we end up with much less:

edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ ls -lh src/RProtoBuf.so 
-rwxr-xr-x 1 edd edd 626K Aug 14 20:29 src/RProtoBuf.so
edd@brad:~/git/rprotobuf(feature/fewer_warnings)$ 

So we reduced the size from 12mb to 0.6mb, an 18-fold decrease. And the file tool still shows the file as 'not stripped' as it still contains the symbols. Only debugging information was removed.

What reduction in size can one expect, generally speaking? I have seen substantial reductions for C++ code, particularly when using tenmplated code. More old-fashioned C code will be less affected. It seems a little difficult to tell---but this method is my new build default as I continually find rather substantial reductions in size (as I tend to work mostly with C++-based packages).

The second option only occured to me this evening, and complements the first which is after all only applicable locally via the ~/.R/Makevars file. What if we wanted it affect each installation of a package? The following addition to its src/Makevars should do:

strippedLib: $(SHLIB)
        if test -e "/usr/bin/strip"; then /usr/bin/strip --strip-debug $(SHLIB); fi

.phony: strippedLib

We declare a new Makefile target strippedLib. But making it dependent on $(SHLIB), we ensure the standard target of this Makefile is built. And by making the target .phony we ensure it will always be executed. And it simply tests for the strip tool, and invokes it on the library after it has been built. Needless to say we get the same reduction is size. And this scheme may even pass muster with CRAN, but I have not yet tried.

Lastly, and acknowledgement. Everything in this post has benefited from discussion with my former colleague Dan Dillon who went as far as setting up tooling in his r-stripper repository. What we have here may be simpler, but it would not have happened with what Dan had put together earlier.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.



Reproducible builds folks: Reproducible Builds: Weekly report #119

August 14, 2017 23:30, by Planet Debian - 0no comments yet

Here's what happened in the Reproducible Builds effort between Sunday July 30 and Saturday August 5 2017:

Media coverage

We were mentioned on Late Night Linux Episode 17, around 29:30.

Packages reviewed and fixed, and bugs filed

Upstream packages:

  • Bernhard M. Wiedemann:
    • efl (merged), unique ids based on memory address
    • 389-ds (merged), SOURCE_DATE_EPOCH support.
    • plowshare, SOURCE_DATE_EPOCH support
    • sphinx, file ordering
    • sphinx, SOURCE_DATE_EPOCH support

Debian packages:

Reviews of unreproducible packages

29 package reviews have been added, 72 have been updated and 151 have been removed in this week, adding to our knowledge about identified issues.

4 issue types have been updated:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (36)
  • Andreas Beckmann (2)
  • Daniel Schepler (2)
  • Logan Rosen (1)
  • Lucas Nussbaum (93)

diffoscope development

Version 85 was uploaded to unstable by Mattia Rizzolo. It included contributions from:

  • Mattia Rizzolo:
    • Add an explicit Recommends: on the defusedxml python package.
    • Various other code quality tweaks.
  • Juliana Oliveira Rodrigues:
    • Fix test_ico_image for ImageMagick identify >= 6.9.8.
    • Use the defusedxml XML library by default in the XML comparator, if it's available. This protects against various XML parser DoS attacks and other security holes, which other Python XML libraries are vulnerable to.
  • Ximin Luo:
    • Force a flush when writing output to diff. (Closes: #870049).

as well as previous weeks' contributions, summarised in the changelog.

There were also further commits in git, which will be released in a later version:

  • Guangyuan Yang:
    • tests/iso9660: support isoinfo's output coming from cdrtools' version instead of genisoimage's
  • Mattia Rizzolo:
    • Code quality and test fixes.
  • Chris Lamb:
    • Code quality and test fixes.

Misc.

This week's edition was written by Ximin Luo, Bernhard M. Wiedemann and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.



Jamie McClelland: Diversity doesn't help the bottom line

August 14, 2017 18:39, by Planet Debian - 0no comments yet

A Google software engineer's sexist screed against diversity has been making the rounds lately.

Most notable are the offensive and mis-guided statements about gender essentialism, which honestly make the thing hard to read at all.

What seems lost in the hype, however, is that his primary point seems quite accurate. In short: If Google successfully diversified it's workforce, racial and gender tensions would increase not decrease, divisiveness would spread and, with all liklihood, Google could be damaged.

Imagine what would happen if the thousands of existing, mostly male, white and Asian engineers, the majority of whom are convinced that they play no part in racism and sexism, were confronted with thousands of smart and ambitious women, African Americans and Latinos who were becoming their bosses, telling them to work in different ways, and taking "their" promotions.

It would be a revolution! I'd love to see it. Google's bosses definitely do not.

That's why none of the diversity programs at Google or any other major tech company are having any impact - because they are not designed to have an impact. They are designed to boost morale and make their existing engineers feel good about what they do.

Google has one goal: to make money. And one strategy: to design software that that people want to use. One of their tactics that is highly effective is building tight knit groups of programmers who work well together. If the creation of hostile, racist and sexist environments is a by-product - well, it's not one that affects their bottom line.

Would Google make better software with a more diverse group of engineers? Definitely! For one, if African American engineers were working on their facial recognition software, it's doubtful it would have mistaken people with black faces for gorillas.

However, if the perceived improvement in software outweighed the risks of diversification, then Google would not waste any time on feel-good programs and trainings - they would simply build a jobs pipeline and change their job outreach programs to recruit substantially more female, African Americans and Latino candidates.

In the end, this risk avoidance and failure to perceive the limitations of homogeneity is the achiles heel of corporate software design.

Our challenge is to see what we can build outside the confines of corporate culture that prioritizes profits, production efficiency, and stability. What can we do with teams that are willing to embrace racial and gender tension, risk diviseveness and be willing to see benefits beyond releasing version 1.0?



Antonio Terceiro: Debconf17

August 14, 2017 17:27, by Planet Debian - 0no comments yet

I’m back from Debconf17.

I gave a talk entitled “Patterns for Testing Debian Packages”, in which I presented a collection of 7 patterns I documented while pushing the Debian Continuous Integration project, and were published in a 2016 paper. Video recording and a copy of the slides are available.

I also hosted the ci/autopkgtest BoF session, in which we discussed issues around the usage of autopkgtest within Debian, the CI system, etc. Video recording is available.

Kudos for the Debconf video team for making the recordings available so quickly!



Holger Levsen: 20170812-reproducible-policy

August 14, 2017 16:53, by Planet Debian - 0no comments yet

"packages should build reproducibly" - after 4 years work this is in debian-policy now

This post was written roughly 44h ago and now that the fix for #844431 has finally marked been merged into the git master branch, I'm publishing it - hoping you'll enjoy this as much as I do!

So today is the last (official) day of DebConf17 and it looks like #844431: "packages should build reproducibly" will be merged into debian-policy today! So I'm super excited, super happy, quite tired and a bit sad (DebConf is ending…) right now! :-)

Four years ago Lunar held a BoF at DebConf13 which started the initiative in Debian. I only got involved in September 2014 with setting up continuous tests, rebuilding each package twice with some variations and then comparing the results using diffoscope, which back then was still called debbindiff and which we renamed as part of our efforts to make Reproducible Builds the norm in Free Software.

Many people have worked on this, and I'm also very happy how visible this has been in our talk here yesterday. You people rock and I'm very thankful and proud to be part of this team. Thank you everyone!

And please understand: we are not 94% done yet (which our reproducibility stats might have made you think), rather more like half done or so. We still need tools and processes to enable anyone to indepently verify that a given binary comes from the sources it is said to be coming, this will involve distributing .buildinfo files and providing user interfaces in APT and elsewhere. And probably also systematic rebuilds by us and other parties. And 6 or 7% of the archive are a lot of packages still, eg in Buster we currently still have 273 unreproducible key packages and for a large part we don't have patches yet. So there is still a lot of work ahead.

This is what was added to debian-policy now:

Reproducibility
---------------

Packages should build reproducibly, which for the purposes of this
document [#]_ means that given

- a version of a source package unpacked at a given path;
- a set of versions of installed build dependencies;
- a set of environment variable values;
- a build architecture; and
- a host architecture,

repeatedly building the source package for the build architecture on
any machine of the host architecture with those versions of the build
dependencies installed and exactly those environment variable values
set will produce bit-for-bit identical binary packages.

It is recommended that packages produce bit-for-bit identical binaries
even if most environment variables and build paths are varied.  It is
intended for this stricter standard to replace the above when it is
easier for packages to meet it.

.. [#]
   This is Debian's precisification of the `reproducible-builds.org
   definition `_.

For now violating this part of policy may result in a severity: normal bug, though I think we should still only file them if we have patches, else it's probably better to just take a note in our notes.git, like we did before the policy change.

Finally one last comment: we could do reproducible security updates for Stretch now too, for those 94% of the packages which are reproducible. It just needs to be done by someones and the first step would be publishing those .buildinfo files from those builds…