% !BIB program = bibtex
\documentclass[sigconf]{acmart}

% No rights block, because this is a draft. Delete these three lines and use
% the values from your rights confirmation email when you submit.
\setcopyright{none}
\settopmatter{printacmref=false}
\renewcommand\footnotetextcopyrightpermission[1]{}

% Your venue. The short name in brackets prints in the page footer.
\acmConference[Conference 2026]{The Conference on Something}{June 1 to 3, 2026}{Somewhere, Country}
\acmYear{2026}
\copyrightyear{2026}
\acmDOI{}
\acmISBN{}

\usepackage{amsmath}
\usepackage{booktabs}

\begin{document}

\title{Title of the paper}

% One \author, \email and \affiliation per person, in that order.
\author{Your Name}
\email{you@example.com}
\affiliation{%
  \institution{University of Somewhere}
  \city{Somewhere}
  \country{Country}
}

\author{Second Author}
\email{second@example.com}
\affiliation{%
  \institution{Other University}
  \city{Othertown}
  \country{Country}
}

\begin{abstract}
% One paragraph: the problem, what you did, what you found.
Developers search their own code far more often than they search the web, yet
code search is usually evaluated with queries borrowed from web search logs. We
analyzed 1.4 million queries issued to the code search tool of a large
engineering organization over six months, and hand coded a sample of 900 of
them. Two thirds of the queries name an identifier the developer already knows
exists, which makes them navigation rather than search, and those queries are
answered in a median of 4 seconds. The remaining third, the queries that
describe behavior instead of naming it, take a median of 71 seconds and are
abandoned four times as often. We describe the coding scheme, report where the
time goes, and argue that the two kinds of query need different interfaces.
\end{abstract}

% Replace this block with the one the ACM CCS tool produces for your paper.
\begin{CCSXML}
<ccs2012>
   <concept>
       <concept_id>10002951.10003317</concept_id>
       <concept_desc>Information systems~Information retrieval</concept_desc>
       <concept_significance>500</concept_significance>
   </concept>
   <concept>
       <concept_id>10011007.10011074</concept_id>
       <concept_desc>Software and its engineering~Software creation and management</concept_desc>
       <concept_significance>300</concept_significance>
   </concept>
</ccs2012>
\end{CCSXML}

\ccsdesc[500]{Information systems~Information retrieval}
\ccsdesc[300]{Software and its engineering~Software creation and management}

\keywords{code search, developer tools, query logs, empirical software engineering}

\maketitle

\section{Introduction}
% What the problem is, why it matters, what this paper adds.
A developer joining a codebase of ten million lines learns it through the
search box. Every question about where a value is set, which caller passes an
empty list, or what a flag was named before it was renamed goes through the
same input field, and the answer arrives in seconds or the developer gives up
and asks a colleague. The tool is therefore one of the most used interfaces in
the organization, and one of the least studied on its own terms.

What little evaluation exists treats code search as a retrieval problem and
measures it with relevance judgments over natural language queries
\cite{sadowski2015}. That framing fits a minority of real traffic. Developers
overwhelmingly type identifiers, and an identifier query has exactly one
correct answer that the developer can recognize instantly, which makes
precision at ten the wrong measure \cite{xia2017}.

This paper reports what the traffic actually looks like. We take six months of
queries from one organization, code a stratified sample by intent, and join
each query to the session that followed it. Section~\ref{sec:method} describes
the data and the coding scheme, Section~\ref{sec:results} reports the split and
its cost in time, and Section~\ref{sec:discussion} draws out what it means for
tool design.

\section{Method}
\label{sec:method}
The log covers 1.4 million queries from 8{,}200 developers over six months.
Each entry holds the query text, a session identifier, the results shown, and
which result was opened, with all identifiers hashed before we saw them.

Two of the authors independently coded a stratified sample of 900 queries into
three intents: \emph{locate}, naming a symbol believed to exist;
\emph{understand}, describing behavior in words; and \emph{explore}, browsing a
subsystem without a target. Agreement was substantial, with Cohen's
$\kappa = 0.81$, and the 47 disagreements were settled by discussion.

Time to answer is measured per session as the interval between the first query
and the last result opened. A session with no result opened within ten minutes
counts as abandoned, following the threshold used in earlier work
\cite{sadowski2015}. Write $t_i$ for the time to answer of session $i$ and
$A$ for the set of sessions that were not abandoned; the median reported below
is taken over $A$ only,
\begin{equation}
  \label{eq:median}
  \tilde{t} = \operatorname{median}_{i \in A} t_i .
\end{equation}

\section{Results}
\label{sec:results}
Table~\ref{tab:intents} gives the split. Locate queries are two thirds of the
traffic and are answered quickly. Understand queries are a third of the traffic
and consume most of the time spent searching, because they usually take several
attempts before the developer finds a word the codebase actually uses.

\begin{table}[t]
  \caption{Query intent, time to answer and abandonment}
  \label{tab:intents}
  \begin{tabular}{lrrr}
    \toprule
    Intent     & Share (\%) & Median (s) & Abandoned (\%) \\
    \midrule
    Locate     & 66 &  4 &  3 \\
    Understand & 29 & 71 & 12 \\
    Explore    &  5 & 38 &  9 \\
    \bottomrule
  \end{tabular}
\end{table}

The gap does not close with experience. Developers in their first month issue
more understand queries than developers with three years in the codebase, as
expected, but the median time to answer an understand query is within four
seconds across every tenure group. The difficulty belongs to the query, not to
the person.

\section{Discussion}
\label{sec:discussion}
Ranking improvements aimed at understand queries are aimed at a third of the
traffic and most of the frustration, which is the right target, but they are
usually evaluated on samples drawn uniformly from the log, where locate queries
drown them out. An evaluation set for code search should be stratified by
intent, and the intent is cheap to predict: a query that parses as an
identifier is a locate query 94 percent of the time.

The threat to this reading is that it comes from one organization with one
language and one tool. The intent split is likely to shift with the language,
since identifier conventions differ, and with whether the tool supports regular
expressions.

\section{Conclusion}
Code search traffic is two different tasks sharing one input field, and only
one of them is retrieval. Separating them costs a regular expression and
changes what the numbers mean.

\bibliographystyle{ACM-Reference-Format}
\bibliography{references}

\end{document}
