A Bugzilla installation (for ELinks) mostly unattended for many years produced a temporary directory where just the directory _itself_ was about 32M big – there must’ve been quite a few million of files. rm -r was able to deal with it, with a small catch – the machine would quickly become totally unusable, with mp3 playback, the mailer and everything stalling for tens of seconds as the I\/O queue would get jammed.<\/p>\n
But the strangest thing is that I found no way to prevent that – at least on my ext3 filesystem and 2.6.29! I tried running rm with nice -n 19, of course that didn’t help. Much more surprisingly, ionice -c 3 or ionice -c 2 -n 7 had no effect either – why is that is lost on me… Changing the I\/O queue governor from the cfq default didn’t help either. In the end, I had to manually ratelimit the removals to one per 1ms… In the end, I used this snippet to delete the directory:<\/p>\n
#define _XOPEN_SOURCE 500\r\n#include <ftw.h>\r\n#include <stdio.h>\r\n#include <stdlib.h>\r\n#include <string.h>\r\n#include <stdint.h>\r\n#include <unistd.h>\r\n\r\nstatic int\r\ndisplay_info(const char *fpath, const struct stat *sb,\r\n int tflag, struct FTW *ftwbuf)\r\n{\r\n puts(fpath);\r\n unlink(fpath);\r\n usleep(1000);\r\n return 0;\r\n}\r\n\r\nint\r\nmain(int argc, char *argv[])\r\n{ \r\n int flags = 0;\r\n \r\n if (nftw((argc < 2) ? \".\" : argv[1], display_info, 20, flags) == -1) {\r\n perror(\"nftw\");\r\n exit(EXIT_FAILURE);\r\n }\r\n exit(EXIT_SUCCESS);\r\n}<\/pre>\nDoes anyone know if Linux really cannot help you prevent a single process totally jamming the I\/O of a machine?<\/p>\n","protected":false},"excerpt":{"rendered":"
A Bugzilla installation (for ELinks) mostly unattended for many years produced a temporary directory where just the directory _itself_ was about 32M big – there must’ve been quite a few million of files. rm -r was able to deal with it, with a small catch – the machine would quickly become totally unusable, with mp3 […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[18],"class_list":["post-33","post","type-post","status-publish","format-standard","hentry","category-linux","tag-kernel"],"_links":{"self":[{"href":"https:\/\/log.or.cz\/index.php?rest_route=\/wp\/v2\/posts\/33","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/log.or.cz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/log.or.cz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/log.or.cz\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/log.or.cz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=33"}],"version-history":[{"count":2,"href":"https:\/\/log.or.cz\/index.php?rest_route=\/wp\/v2\/posts\/33\/revisions"}],"predecessor-version":[{"id":35,"href":"https:\/\/log.or.cz\/index.php?rest_route=\/wp\/v2\/posts\/33\/revisions\/35"}],"wp:attachment":[{"href":"https:\/\/log.or.cz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=33"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/log.or.cz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=33"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/log.or.cz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=33"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}