Hey,

These days I’ve been doing some research that involves looking at the Linux kernel to figure things out.

Since I’ve been using YouCompleteMe for a reasonable amount of time, and that I love how simple it is to configure it, why not give a try to using it for inspecting the Linux kernel?

Example of YouCompleteMe searching available functions

Example of YouCompleteMe searching available functions.

ps.: for a browser experience, bootlin is AMAZING. It lets you navigate through the code providing links to definitions and making you go back in time to previous versions of the source - it’s great! Here I focus on navigating the Kernel source using VIM and a terminal though.

Check out how to tailor a YouCompleteMe configuration that suits the Linux kernel source code.

Getting YouCompleteMe ready

The first thing to do is getting YCM ready.

As I’m constantly doing some experiments in AWS and from time to time running a VM here and there, I automated the process using Ansible and Packer (with Ansible I describe how to get to the desired state, with Packer I build a base image) - check out cirocosta/mylinux.

Below is how the YouCompleteMe part of it looks like:

- name: 'install ycm'
  command: './install.py --go-completer --clang-completer'
  environment: 
    PATH: "{{ ansible_env.PATH }}:/usr/local/go/bin"
  args:
    chdir: '{{ user_home }}/.vim/bundle/YouCompleteMe'

ps.: You might notice that I also specify --go-completer there. That’s not needed for C code (like Linux), so you can get rid of that.

pps.: Check out my Vim Ansible role to see the full installation of my Vim setup.

As I use pathogen to provide Vim plugins to my environment, there’s just the need to clone the repository to .vim/bundle (with submodules) and then running the installation script.

Setting up the YouCompleteMe configuration

Given that the Kernel does not make use of CMake or Ninja, we can’t make use of the compilation database that CMake would create (a database that tells which commands were used to compile a given file).

Update (30 Jun, 2018): it turns out that you can get a full fidelity compilation database even without cmake - the bear utility takes the approach of intercepting the build calls, gather the relevant info and generate a complete compilation database. Having such database ready, you can either make full use of it and ignore an extra .ycm_extra_conf.py or gather flags you find useful from there and tailor your special config. Thanks lanzaio!

However, we can still have such mapping by providing a .ycm_extra_conf.py file that does the job of delivering the compilation flags for any given file. The whole point of this python file is to make the project owner implement the YCM interface for detecting flags for files:

Given a source file, give me its compilation flags.

As it’s common in C project to have a “catch all” target in a Makefile (like %.o: %.c) that provides the same flags to every file, we can have a pretty simplified .ycm_extra_conf.py file:

def FlagsForFile(filename, **kwargs):
  return { 'flags': ['-I/usr/include', '-std=gnu89', '-xc'] }

For Linux, that’s not very different - gather the flags used, supply them to FlagsForFile and we’re done.

The necessary flags can be gathered by exposing two variables that get set in the Makefile: LINUXINCLUDE and USERINCLUDE. That’s because, for just autocompletion and source code navigation, includes and definitions are sufficient.

To dump those variables, create an extra target in ./Makefile:

diff --git a/Makefile b/Makefile
index 019a5a02..88e1ad10 100644
--- a/Makefile
+++ b/Makefile
@@ -1792,6 +1792,10 @@ endif    # skip-makefile
 PHONY += FORCE
 FORCE:
 
+show-includes:
+       @$(foreach include, $(LINUXINCLUDE), echo $(include);)
+       @$(foreach include, $(USERINCLUDE), echo $(include);)
+
 # Declare the contents of the .PHONY variable as phony.  We keep that
 # information in a v

Then prepare the .config file that would be necessary for the tiniest possible kernel:

make tinyconfig

With the configuration set, now, executing make specifying the show-includes target dumps the variables we need:

# Piping it to `sort -u` to remove the duplicates
# and keep the list sorted.
make show-includes | sort -u
-I./arch/x86/include
-I./arch/x86/include/generated
-I./arch/x86/include/generated/uapi
-I./arch/x86/include/uapi
-I./include
-I./include/generated/uapi
-I./include/uapi
-include
./include/linux/kconfig.h

As the next step is to supply those variables to YouCompleteMe, we need first to realize that the daemon (ycmd) will not deal well with these relative paths.

In our Python script we can deal with that though:

import os

CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))

include_dirs = [
    './arch/x86/include',
    './arch/x86/include/generated',
    './arch/x86/include/generated/uapi',
    './arch/x86/include/uapi',
    './include',
    './include/generated/uapi',
    './include/uapi',
]


include_files = [
    './include/linux/kconfig.h',
]

flags = [
    '-D__KERNEL__',
    '-std=gnu89',
    '-xc',
    '-nostdinc',
]

def FlagsForFile( filename, **kwargs ):
    """
    Given a source file, retrieves the flags necessary for compiling it.
    """
    for dir in include_dirs:
        flags.append('-I' + os.path.join(CURRENT_DIR, dir))

    for file in include_files:
        flags.append('-include' + os.path.join(CURRENT_DIR, file))

    return { 'flags': flags }

Now, open a file (like, ./net/socket.c) and check the autocompletion working! You can also check how “jump to” declarations and definitions also work.

Example of YouCompleteMe autocompleting a kernel struct

Example of YouCompleteMe autocompleting a kernel struct.

If you have any questions or found any error, please let me know! I’m cirowrc on Twitter.

Have a good one!