From MAILER-DAEMON Wed Oct 26 19:41:12 2022 Received: from list by lists.fsf.org with archive (Exim 4.90_1) id 1onq1I-000125-0k for mharc-tech-volunteer-meeting@fsf.org; Wed, 26 Oct 2022 19:41:12 -0400 Received: from mail.fsf.org ([2001:470:142::13]) by lists.fsf.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1onq1E-00010m-5d for tech-volunteer-meeting@fsf.org; Wed, 26 Oct 2022 19:41:08 -0400 Received: from havoc.proulx.com ([96.88.95.61]) by mail.fsf.org with esmtps (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.93) (envelope-from ) id 1onq10-00DeU7-Tk for tech-volunteer-meeting@fsf.org; Wed, 26 Oct 2022 19:41:03 -0400 Received: from joseki.proulx.com (localhost [127.0.0.1]) by havoc.proulx.com (Postfix) with ESMTP id 98223492 for ; Wed, 26 Oct 2022 17:40:48 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proulx.com; s=dkim2048; t=1666827648; bh=2MhF47d6nDUwxI24872L4J4SKaI+U4K3WO4I1lbsP4Q=; h=Date:From:To:Subject:From; b=nBGiV0SUWqTMKlquP/DrkXLfETsX91rWuqwrEpFsIBqqOZWN05iqG3OQebxitHrHs uaaS1TbSvxiX7zr5Hi7Ck0pUCXLcOniDTbEPouQQqbcVaWUM3kOtU4vt871DW6BrnD dqg5xmqMMHdaGcU79NdNMza/P+6xZunKUTdHre38Z/+T98MqU43EugLGiD2EMaVpmz EguDuN0NZFBm6u0Aovye5g7zji6vIuoGYOtMx9xFSVNCF/5lyNYDZ5aNvtOcITke63 tqSfbACAURYNymcgyLj71ObQFWAuhOWZKGl3lbxiD4HHoZQSAOTgOJIdkhWmkZaJz8 4duyossbjgMEQ== Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 7322A7A018 for ; Wed, 26 Oct 2022 17:40:48 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 6989469262A8; Wed, 26 Oct 2022 17:40:48 -0600 (MDT) Date: Wed, 26 Oct 2022 17:40:48 -0600 From: Bob Proulx To: tech-volunteer-meeting@fsf.org Subject: Upgrade interesting dependency tidbit Message-ID: <20221026173355417840495@bob.proulx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Received-SPF: pass client-ip=96.88.95.61; envelope-from=bob@proulx.com; helo=havoc.proulx.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: tech-volunteer-meeting@fsf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Oct 2022 23:41:09 -0000 This is not anything important. It's just something that I thought was somewhat interesting. I know that 99.44% of people won't find it interesting though. I share it anyway. :-) After upgrading to the latest OS release I went through the cleaning steps that I outlined in a previous mail. One of them is that I look at the "deborphan" output which lists libraries that are installed that nothing depends upon. Purging those off. And then repeating. Repeating. Repeating. Will I never hit the end of the chain? I did that here. Look at the depth of that dependency chain! Wow! root@frontend2:/etc# deborphan libidn11 libldap-2.4-2 libmpdec2 libssl1.1 root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49335 files and directories currently installed.) Removing libidn11:amd64 (1.33-3build1) ... Removing libldap-2.4-2:amd64 (2.4.49+dfsg-2ubuntu1.9) ... Removing libmpdec2:amd64 (2.4.2-3) ... Removing libssl1.1:amd64 (1.1.1l-1ubuntu1) ... Purging configuration files for libssl1.1:amd64 (1.1.1l-1ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... root@frontend2:/etc# deborphan libgssapi3-heimdal root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49306 files and directories currently installed.) Removing libgssapi3-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... root@frontend2:/etc# deborphan libheimntlm0-heimdal root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49300 files and directories currently installed.) Removing libheimntlm0-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... root@frontend2:/etc# deborphan libkrb5-26-heimdal root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49294 files and directories currently installed.) Removing libkrb5-26-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... root@frontend2:/etc# deborphan libhx509-5-heimdal root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49288 files and directories currently installed.) Removing libhx509-5-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... root@frontend2:/etc# deborphan libhcrypto4-heimdal libwind0-heimdal root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49282 files and directories currently installed.) Removing libhcrypto4-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Removing libwind0-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... root@frontend2:/etc# deborphan libasn1-8-heimdal libheimbase1-heimdal root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49270 files and directories currently installed.) Removing libasn1-8-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Removing libheimbase1-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... root@frontend2:/etc# deborphan libroken18-heimdal root@frontend2:/etc# dpkg --purge $(deborphan) (Reading database ... 49258 files and directories currently installed.) Removing libroken18-heimdal:amd64 (7.7.0+dfsg-3ubuntu1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1+11.0trisquel1) ... That is quite the dependency chain of libraries depending upon libraries depending upon libraries! And at the very bottom someone has named the library "libroken18-heimdal" as in broken? We do live in interesting times. Bob From MAILER-DAEMON Sat Oct 29 15:17:39 2022 Received: from list by lists.fsf.org with archive (Exim 4.90_1) id 1oorKt-0005dG-8n for mharc-tech-volunteer-meeting@fsf.org; Sat, 29 Oct 2022 15:17:39 -0400 Received: from mail.fsf.org ([2001:470:142::13]) by lists.fsf.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oorKr-0005d4-Hy for tech-volunteer-meeting@fsf.org; Sat, 29 Oct 2022 15:17:37 -0400 Received: from havoc.proulx.com ([96.88.95.61]) by mail.fsf.org with esmtps (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.93) (envelope-from ) id 1oorKp-002Aog-1L for tech-volunteer-meeting@fsf.org; Sat, 29 Oct 2022 15:17:37 -0400 Received: from joseki.proulx.com (localhost [127.0.0.1]) by havoc.proulx.com (Postfix) with ESMTP id A35F733F for ; Sat, 29 Oct 2022 13:17:30 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proulx.com; s=dkim2048; t=1667071050; bh=qQLvb+q51PfFHeQhIE8PO8FNqCBVJymp86z7eeTk3xM=; h=Date:From:To:Subject:From; b=IeEQwhN8tydXbU6hgy1NjkXsFjbDRya1PlHadvl2Vop/XonYNQb0ogSstlPj5mU28 hJciTcH9/puHA2lQFOZshQuCwMxxhLTV4gDro4GUJm5VIkz5Zck8deEqyQwGWq6vM4 w8oxlGIFOze0PQ1nVuL4pqkwT5GnYwbyK6bEe2cyfNX8i1p2EoKWWo9r5lpF2I2c0+ otOcBjJObv65dAxbWPh5oN1OG3fzLJ/cpmVnXFuANn4tSd3lHVmo2gm5C5AcTM1xoJ 2liYROiQEW7CrrEaD0eJ8Kgde6IVLAmvfZ3d9/vEQ/vpAsqB7dRjxwykH+kMpywpZe OPc/OoQ30zx3g== Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id 7F5437A018 for ; Sat, 29 Oct 2022 13:17:30 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 6B8D76925950; Sat, 29 Oct 2022 13:17:30 -0600 (MDT) Date: Sat, 29 Oct 2022 13:17:30 -0600 From: Bob Proulx To: tech-volunteer-meeting@fsf.org Subject: Google needs slow delivery Message-ID: <20221029124042235253378@bob.proulx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Received-SPF: pass client-ip=96.88.95.61; envelope-from=bob@proulx.com; helo=havoc.proulx.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: tech-volunteer-meeting@fsf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Oct 2022 19:17:37 -0000 Hi Ian, I use Postfix so not sure how much of this will translate to Exim but if I do not slow down delivery to Google then I get rejections from Google. Because even though Google accounts are a huge source of spam to me Google also is very parsnickety about accepting mail. Google is the worst of them for me. Probably because so many people use Gmail that it is the one I run into delivery problems the most. But certainly Yahoo and especially Microsoft cause me other specific pain. But this is what I need to do for Google. In /etc/postfix/master.cf where services are defined configure an addtional Gmail specific service. I use a custom logging name and therefore override the logging name for this service. # Google rate limits incoming mail to the point of failure. # Use a Gmail specific transport which is slow. gmail unix - - y - - smtp -o syslog_name=postfix/gmail In the main.cf file configure custom rate limits for this service. In Postfix the first part of the variable name matches the service name and the later part of the name matches the custom variable configured for that service. So in this case gmail_destination_concurrency_limit configures "gmail" for "gmail_destination_concurrency_limit" of 2. And so on for the rest. gmail_destination_concurrency_limit = 2 gmail_destination_rate_delay = 1s gmail_destination_recipient_limit = 2 The defaults from postfix upstream are these. default_destination_concurrency_limit = 20 default_destination_rate_delay = 0s default_destination_recipient_limit = 50 Normally one might *increase* those in order to speed delivery. But with Google I have had to decrease those in order to slow delivery. Otherwise if I send to three people that are all at gmail.com then I get a reject notice. It's totally insane. In the /etc/postfix/transport map file define a mapping from @gmail.com to this transport. Run "postmap transport" to rebuild the transport.db database file. # Google rate limits incoming mail to the point of failure. # Use a Gmail specific policy which is slow. gmail.com gmail In Postfix most of the time restarting is not necessary. If changing the master.cf file a "postfix reload" is needed to instruct the supervisor process that options have changed. The rest is automatic. Which is to say that for the most part Postfix can be continuously running with no perceptable downtime most of the time. Google appears to group delivery by domain name even if they are handling multiple domains. So this only seems to affect @gmail.com addresses. I have a friend with a custom domain that is handled by Gmail through the standard MX records to go there directly. There is only one address there. I never have reject problems with that person's one email address. *Even though the content of the message is identical*. Google will reject the three to @gmail.com and accept the message to the other domain which is going to the same server. I did not think up the above configuration myself. I read about the technique here and then implemented it. The example is why I call this the turtle transport. The main link is dead now. Shout-Out to the Internet Archive! https://web.archive.org/web/20130628013524/http://steam.io/2013/04/01/postfix-rate-limiting/ Here is the main Postfix documentation page. https://www.postfix.org/documentation.html Just by way of kickstarting the documentation the overview is here. http://www.postfix.org/OVERVIEW.html But for this topic probably the specific documentation is here. http://www.postfix.org/TUNING_README.html#rope That last is the specific documentation on this topic. But I wanted to sneak up on it with the more general overview stuff first. Since otherwise it's learning to swim by jumping into the deep end. Hope this helps! Bob From MAILER-DAEMON Sat Oct 29 15:39:40 2022 Received: from list by lists.fsf.org with archive (Exim 4.90_1) id 1oorgC-0006mp-69 for mharc-tech-volunteer-meeting@fsf.org; Sat, 29 Oct 2022 15:39:40 -0400 Received: from mail.fsf.org ([2001:470:142::13]) by lists.fsf.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oorgA-0006mb-6o for tech-volunteer-meeting@fsf.org; Sat, 29 Oct 2022 15:39:38 -0400 Received: from havoc.proulx.com ([96.88.95.61]) by mail.fsf.org with esmtps (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.93) (envelope-from ) id 1oorg8-002Co4-5l for tech-volunteer-meeting@fsf.org; Sat, 29 Oct 2022 15:39:37 -0400 Received: from joseki.proulx.com (localhost [127.0.0.1]) by havoc.proulx.com (Postfix) with ESMTP id C4EBC33F for ; Sat, 29 Oct 2022 13:39:34 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proulx.com; s=dkim2048; t=1667072374; bh=uDfM86q+6Mc0/44L47nfTT6e9ZpUsDwHr57p+u5v5jY=; h=Date:From:To:Subject:From; b=MtIEcYwPjOsH4lH81RxN28zFHlYPbZUmtStStl30ZQ3UPDc7p1kQcnks+MGiIxGuB fkrFlIedFIihR5Q+/CppXNMtpvT+THJzeQy/lcYDom2As+8gV1IrQhxpP4FmE9cdXL i7oaZm32Xf05IWIJWt/QiPwonWYts0wieHWkc7Qhh1/VmGPGLin8/zc7MW4gz3Kx5T 0Fd9tnAMaQuISK0nOM/Lh4JBHklsxeBJaXHZefmqo81dKduyQEiilnF5niMGQ68ynL t/QmlKYFk/CfI8+dzACoVwfTTRJ4hvKZjDi1Mh1wavBQlnUuaFCsy+hzGAVWKGZ4Y0 Jr9xNd0j+Dypw== Received: from hysteria.proulx.com (hysteria.proulx.com [192.168.230.119]) by joseki.proulx.com (Postfix) with ESMTP id A51217A018 for ; Sat, 29 Oct 2022 13:39:34 -0600 (MDT) Received: by hysteria.proulx.com (Postfix, from userid 1000) id 9ACA16925950; Sat, 29 Oct 2022 13:39:34 -0600 (MDT) Date: Sat, 29 Oct 2022 13:39:34 -0600 From: Bob Proulx To: tech-volunteer-meeting@fsf.org Subject: Trisuel AIDE Package Upgrade Snag And Rant Message-ID: <20221029132100251731327@bob.proulx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Received-SPF: pass client-ip=96.88.95.61; envelope-from=bob@proulx.com; helo=havoc.proulx.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: tech-volunteer-meeting@fsf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Oct 2022 19:39:38 -0000 The recent Trisquel 10 to 11 upgrade caught a small snag for the AIDE package upgrade. This might be considered a daily rant. AIDE -- Advanced Intrusion Detection Environment. It is useful to notify in the case of changes to system files. This is one of the standard packages which does this. Just one cog in the machine. I upgraded frontend2 to the latest Trisuel 11 release candidate (equiv to Ubuntu 22.04) and hit another one of those times when you can't believe that it is completely broken and hasn't been fixed. These packages of aide and aide-common should be passthrough packages from Ubuntu into Trisquel so probably broken in Ubuntu too but don't know as I didn't verify it there. This is the Trisquel 10 equiv Ubuntu 20.04 version of the file. root@frontend1:~# ll /etc/aide/aide.conf.d/31_aide_smokeping -rwxr-xr-x 1 root root 476 Dec 16 2013 /etc/aide/aide.conf.d/31_aide_smokeping root@frontend1:~# cat /etc/aide/aide.conf.d/31_aide_smokeping #!/bin/bash if [ -d "/var/lib/smokeping" ]; then find /var/lib/smokeping -type f -name '*.rrd' | \ sed 's/^\(.*\)/\1$ VarFile/' fi if [ -d "/var/www/smokeping" ]; then find /var/www/smokeping -type f -name '*.png' | \ sed 's/^\(.*\)/\1$ VarFile/' find /var/www/smokeping -type f -name '*.maxhight' | \ sed 's/^\(.*\)/\1$ VarFile/' fi cat < ERROR: /etc/aide/aide.conf.d/31_aide_smokeping: execl failed: Exec format error ERROR: /etc/aide/aide.conf.d/31_aide_smokeping: execution failed (exit status: 1) AIDE --init return code 20 I look in the aide-common.postinst script and find this snippet. # in aide 0.17.2-1, the smokeping snippet has been made static # instead of being a executable snippet. Remove x bits on an unchanged # file if [ "$(sha256sum /etc/aide/aide.conf.d/31_aide_smokeping)" = "99f11d04cf33318cef28cb71c252ee0f7b5c37e91893e813973ee120f301a5a7 /etc/aide/aide.conf.d/31_aide_smokeping" ]; then chmod 0644 /etc/aide/aide.conf.d/31_aide_smokeping fi root@frontend1:~# dpkg -l |grep aide ii aide 0.16.1-1ubuntu0.1 amd64 AIDE - static binary ii aide-common 0.16.1-1ubuntu0.1 all AIDE - Common files root@frontend2:~# dpkg -l |grep aide ii aide 0.17.4-1 amd64 AIDE - dynamic binary ii aide-common 0.17.4-1 all AIDE - Common files I abbreviated this above slightly. root@frontend2:~# sha256sum /etc/aide/aide.conf.d/31_aide_smokeping 38da62161e480becd5eac8373058bbe25f78a2cfe46e116aebe44cf1060393f5 /etc/aide/aide.conf.d/31_aide_smokeping root@frontend1:~# sha256sum /etc/aide/aide.conf.d/31_aide_smokeping cdd755947d2847e96290b675f95fd2b67f13a5e861f49c0d4e6bbc3bf6d18923 /etc/aide/aide.conf.d/31_aide_smokeping It was 0.16.1-1ubuntu0.1 in the previous version. So they are assuming that this will match that particular signature. But it doesn't. That's the bug. Fix it manually. root@frontend2:~# chmod a-x /etc/aide/aide.conf.d/31_aide_smokeping I look at a Debian system and it has the right file hash. root@despair:~# sha256sum /etc/aide/aide.conf.d/31_aide_smokeping 99f11d04cf33318cef28cb71c252ee0f7b5c37e91893e813973ee120f301a5a7 /etc/aide/aide.conf.d/31_aide_smokeping despair: ii aide 0.17.3-4+deb11u1 i386 AIDE - static binary despair: ii aide-common 0.17.3-4+deb11u1 all AIDE - Common files So the problem is that Ubuntu took the Debian package directly. Which is usually okay. Except that the OS releases have different release points containing different versions from Ubuntu. Effectly Ubuntu, and therefore Trisquel, skipped the Debian release containing the file with exactly that file content hash signature. And as we all know it is not supported to skip releases! This is one example of why. I don't want to bury this in the noise. I'll say it again to emphasize. Trisquel skips a release point. And skipping release points is not supported. Trisquel should have forked for a different hash signature in the postinst script. Or even better improved the postinst script. Instead of the rigid file checksum signature they could have looked to see if it was the previous shell script or not. root@frontend1:~# sed 1q /etc/aide/aide.conf.d/31_aide_smokeping #!/bin/bash root@frontend2:~# sed 1q /etc/aide/aide.conf.d/31_aide_smokeping @@define VLS var/lib/smokeping And then base the action upon it being a script or not. Because any change will break the hash signature. This snag was hit with pristine copies of the file. It was NOT modified locally. There is the possible issue that someone might want the file to be broken, might have modified the file, and might argue that as a local change the postinst script should not change the file mode in the more flexible method I describe. I only know one person that pedantic who routinely argues this point and I think the greater good out weighs. The postinst author simply created a rigid system, rigid systems are fragile, and it broke due to lacking flexibility. This is an example of why rigid systems are bad and more flexible systems are better. The crazy thing is that it is still broken? Ubuntu upgrades are always like moving from one unfinished house into another unfinished house. This has been, A Daily Rant! Bob