{"id":896,"date":"2024-10-06T20:12:57","date_gmt":"2024-10-06T18:12:57","guid":{"rendered":"https:\/\/www.dereckson.be\/blog\/?p=896"},"modified":"2025-09-27T03:05:17","modified_gmt":"2025-09-27T01:05:17","slug":"openmetrics-for-ccache","status":"publish","type":"post","link":"https:\/\/www.dereckson.be\/blog\/2024\/10\/06\/openmetrics-for-ccache\/","title":{"rendered":"OpenMetrics for ccache"},"content":{"rendered":"\n<p>The <a href=\"https:\/\/ccache.dev\/\" data-type=\"URL\" data-id=\"https:\/\/ccache.dev\/\">ccache<\/a> software maintains a cache to store compilation artefacts, so the next time you compile the same source file, you can get the compiled code from the cache. That speeds up a lot recompilation.<\/p>\n\n\n\n<p>At Nasqueron, ccache is useful to speed up <a href=\"https:\/\/github.com\/freebsd\/poudriere\" data-type=\"URL\" data-id=\"https:\/\/github.com\/freebsd\/poudriere\">Poudriere<\/a> builds to test FreeBSD ports. To be able to follow if the cache is useful, we devised a Grafana dashboard to expose metrics from ccache.<\/p>\n\n\n\n<p>If you run <code>ccache -s<\/code>, it <a href=\"https:\/\/ccache.dev\/manual\/4.10.2.html#_cache_statistics\">prints statistics<\/a>. For example with ccache 3:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$ ccache -V | head -n1\nccache version 3.7.12\n\n$ ccache -s\ncache directory                     \/var\/cache\/ccache\nprimary config                      \/var\/cache\/ccache\/ccache.conf\nsecondary config      (readonly)    \/usr\/local\/etc\/ccache.conf\nstats updated                       Sun Oct  6 13:56:59 2024\ncache hit (direct)                   571\ncache hit (preprocessed)             309\ncache miss                         10089\ncache hit rate                      8.02 %\ncalled for link                     1577\ncalled for preprocessing             265\nmultiple source files                  3\ncompiler produced no output            3\ncompiler produced empty output         4\ncompile failed                       612\npreprocessor error                   292\nbad compiler arguments                68\nunsupported source language           14\nautoconf compile\/link               2829\nno input file                        426\ncleanups performed                     0\nfiles in cache                     24590\ncache size                         167.9 kB\nmax cache size                       5.0 GB<\/code><\/pre>\n\n\n\n<p>Converted into OpenMetrics format, that gives:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>$ ccache-metrics\n# TYPE ccache_cache_directory info\nccache_cache_directory_info{path=\"\/var\/cache\/ccache\"} 1\n# TYPE ccache_primary_config info\nccache_primary_config_info{path=\"\/var\/cache\/ccache\/ccache.conf\"} 1\n# TYPE ccache_secondary_config info\nccache_secondary_config_info{path=\"\/usr\/local\/etc\/ccache.conf\"} 1\n# TYPE ccache_stats_updated counter\nccache_stats_updated_total 1728223019\n# TYPE ccache_cache_hit_direct counter\nccache_cache_hit_direct_total 571\n# TYPE ccache_cache_hit_preprocessed counter\nccache_cache_hit_preprocessed_total 309\n# TYPE ccache_cache_miss counter\nccache_cache_miss_total 10089\n# TYPE ccache_cache_hit_rate gauge\nccache_cache_hit_rate 0.0802\n# TYPE ccache_called_for_link counter\nccache_called_for_link_total 1577\n# TYPE ccache_called_for_preprocessing counter\nccache_called_for_preprocessing_total 265\n# TYPE ccache_multiple_source_files counter\nccache_multiple_source_files_total 3\n# TYPE ccache_compiler_produced_no_output counter\nccache_compiler_produced_no_output_total 3\n# TYPE ccache_compiler_produced_empty_output counter\nccache_compiler_produced_empty_output_total 4\n# TYPE ccache_compile_failed counter\nccache_compile_failed_total 612\n# TYPE ccache_preprocessor_error counter\nccache_preprocessor_error_total 292\n# TYPE ccache_bad_compiler_arguments counter\nccache_bad_compiler_arguments_total 68\n# TYPE ccache_unsupported_source_language counter\nccache_unsupported_source_language_total 14\n# TYPE ccache_autoconf_compile_link counter\nccache_autoconf_compile_link_total 2829\n# TYPE ccache_no_input_file counter\nccache_no_input_file_total 426\n# TYPE ccache_cleanups_performed counter\nccache_cleanups_performed_total 0\n# TYPE ccache_files_in_cache gauge\nccache_files_in_cache 24590\n# TYPE ccache_cache_size gauge\nccache_cache_size 171929.6\n# TYPE ccache_max_cache_size gauge\nccache_max_cache_size 5368709120.0\n# EOF<\/code><\/pre>\n\n\n\n<p>Those metrics can then be published to a timeseries database like Prometheus.<\/p>\n\n\n\n<p>The Python code to perform this conversion is <a href=\"https:\/\/devcentral.nasqueron.org\/D3492#change-HIAnCwboNuWh\" data-type=\"URL\" data-id=\"https:\/\/devcentral.nasqueron.org\/D3492#change-HIAnCwboNuWh\">available in our operations repository<\/a>. It doesn&#8217;t have any dependency.<\/p>\n\n\n\n<p><strong>Important note about ccache versions.<\/strong> As we target FreeBSD and Poudriere, the code handles ccache 3. The parsing code need to be updated for ccache 4, as their statistics output format changed. Run <code>ccache -svv<\/code> to get the same kind of output under ccache 4, but we need to ignore section headers.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The ccache software maintains a cache to store compilation artefacts, so the next time you compile the same source file, you can get the compiled code from the cache. That speeds up a lot recompilation. At Nasqueron, ccache is useful to speed up Poudriere builds to test FreeBSD ports. To be able to follow if [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":959,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[49,50],"tags":[374,9],"class_list":["post-896","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dev","category-sysadmin","tag-ccache","tag-freebsd"],"_links":{"self":[{"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/posts\/896","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/comments?post=896"}],"version-history":[{"count":5,"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/posts\/896\/revisions"}],"predecessor-version":[{"id":902,"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/posts\/896\/revisions\/902"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/media\/959"}],"wp:attachment":[{"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/media?parent=896"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/categories?post=896"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dereckson.be\/blog\/wp-json\/wp\/v2\/tags?post=896"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}