\chapter{Background}

\section{OpenFlow}
OpenFlow, a communications protocol for SDN architecture, consists of
communications messages and mechanisms enabling packet processing for OpenFlow
switches and controllers. The packet processing  mechanism is based on flow
policy, which is a combination of match fields and instruction sets.  The flow
policy is implemented as flow entry and stored in the flow table. The whole
process for packet processing is illustrated in Figure \ref{fig:of_workflow}.
As the packet arrives, the switch tries to match the flow entries in the table
of multiple stages. For each matching process, an action set (with the default of
empty) of the particular action is applied to the packet. Once the packet is matched
with the entry in the table, the action set will be updated and possibly
directed to other tables in a pipelined fashion. Finally, the action set will
be executed at the end of the pipeline process.

\begin{figure}[H]
    \begin{center}
        \includegraphics[width=0.8\textwidth]{OFWorkflow.png}
        \caption{OpenFlow pipeline processing}
        \label{fig:of_workflow}
    \end{center}
\end{figure}

The flow entry, which is the most important part of the process, consists of
priority, counters, match fields, timeouts, cookies, and instructions. The
priority affects the executing order of flow entries. The timeouts contain
timers of hard-timeout and idle-timeout values for flow entry removal when
expired. The match fields are a set of records consist of packet headers
ranging from layer one to four. And the instruction set contains operations to
be executed for the incoming packet which is matched with the match fields.

The available instructions and executing
order are listed in Table \ref{table:of_instructions}.

\begin{center}
    \begin{table}[h]
        \centering
        \begin{tabular}{| l | l | l |}
            \hline
            Instruction     & Description                               & Executing Priority \\ \hline
            Meter           & Apply the specified metering to packet.   & 6 \\ \hline
            Apply-Actions   & Apply actions immediately to packets.     & 5 \\ \hline
            Clear-Actions   & Clear action set.                         & 4 \\ \hline
            Write-Actions   & Write actions to action set.              & 3 \\ \hline
            Write-Metadata  & Write the masked metadata value.          & 2 \\ \hline
            Goto-Table      & Execute the table pipeline.               & 1 \\ \hline
            \hline
        \end{tabular}
        \caption{OpenFlow instruction with executing priority}
        \label{table:of_instructions}
    \end{table}
\end{center}

\section{OpenFlow performance parameters}
%Comment: add 1 paragraph to explain the TD part. and address the OF performance
In the past, the performance testing focuses on the TD part which includes
latency, loss, buffer size, throughput, jitter. However, the most important
factor of performance testing for OpenFlow switch should be the interactions
between the control plane and the data plane. And considering the extra
structures or mechanisms for OpenFlow, we summarize the OpenFlow performance
parameters with C2D, D2C, and OFD parts.

In the C2D part, there are many messages, which include the Flow-mod,
configuration, and state query  of the switch, sent from the controller to the
switches. The switches  report messages  of some events back to the controller
in the D2C part. These messages consist of Packet-in, Flow-removed, status
report, and errors. Therefore, these messages could be treated as the
important parameters  for OpenFlow performance measurement. Moreover, in most
of the cases, the D2C2D process are often executed for the new type of flows. The
process is consisting message transactions of the Packet-in, Packet-out, and
Flow-mod.

The rate of Packet-in message generated by OpenFlow switch is an important
capability indicator   handling the new type of flows. If the rate is low, the
OpenFlow switch will not be able to handle a large number of new flows in a
short period of time. As for the  Packet-out message, it is used to execute
the actions for the Packet-in message in the D2C2D process for  the first
packet of a new flow. The rate of Packet-out message, which is generated by the
controller, reflects the capability of packet processing for OpenFlow switch.
Finally, the duration of how long the flow entry remains active is set by the
Flow-mod message sent to the switch. Therefore, these three messages are the
major parameters of OpenFlow performance measurement for the C2D and the D2C
parts.

In the OFD part, we summarize the  performance parameters in two categories: the
packet processing and OpenFlow mechanism. The former consists of the table
lookup, table pipeline, and time of action set execution. The later contains
the timeout of flow entry and the size of the flow table.

\section{Related Work}
Currently, there are some testing solutions developed with the focus on TD. Bianco
\cite{Bianco2010} and Emmerich \cite{Emmerich2014} provide the measurement methodologies for the forwarding
throughput and packet latency of switch
in underloaded and overloaded situations. Gelberger \cite{Gelberger2013} proposes
the cost measurement which includes throughput, latency,
and jitter for switching and routing on OpenFlow. Jarschel \cite{Jarschel2011} propose the model based on forwarding
speed and blocking probability to estimate the packet sojourn time.

Recently, Spirent proposes a white paper \cite{Spirent_whitepaper} for
OpenFlow switch performance testing. It focuses on the general concepts of
testing and provides suggestions for each test cases. More specifically, the
authors in OFLOPS \cite{OFLOPS} propose two major test cases for the latency
measurement of Flow-mod and the execution time of action set.

Although the latency can be evaluated by the Barrier messages, however, the
measurement result is not accurate where deviation may occur. There are two
major reasons. The one is that the Barrier messages need extra time to be
processed between the controller and the switch. The other is that the process
of Barrier messages in OpenFlow switch may not be implemented correctly. The
methods proposed by OFLOPS \cite{OFLOPS} solve these issues by using the traffic
to measure the setup time of multi-flow entries. In advance, the authors in OFLOPS
propose the OFLOPS-Turbo \cite{OFLOPS-Turbo} which enhances the framework by 10
Gbps Ethernet platform to evaluate the next-generation OpenFlow switch. However,
the system does not address
any methodologies for switch testing in detail.

Due to the fact where latency of Flow-mod may be varied under different testing situations,
Handfield \cite{SDN_Flow_Tables} finds out the major impact factors which include the priority,
current flow table size, and the operation of Flow-mod.
Bianco \cite{Bianco2010} and Tanyingyong \cite{Tanyingyong2010} estimate the
effects of table lookup for hash and linear flow. For the throughput, both of them
\cite{Bianco2010}\cite{Tanyingyong2010} find out the hash flow
have better performance. For the processing time, Bianco \cite{Bianco2010} estimate
it for hash and linear flows with different table size. The results are constant
and independent of the table size for both types. The author
of High-fidelity switch models \cite{Huang2013} proposed a model to emulate the
vendor hardware switches including the Flow-mod rate without stating
the methodologies.

Obviously, as summarized in Table \ref{table:related_work}, the existing
works of literature fail to cover all of the performance parameters needed to evaluate the OpenFlow
switch under test. Therefore, in this thesis, we propose how to
measure all of the major performance parameters except the latency of Flow-mod
and the size of the flow table.

For the size of the flow table measurement, we can
simply use a basic Flow-mod with an add operation. For the
measurement of Packet-in and Packet-out rate, we propose  to evaluate them by
buffer size test case in the process. For the  measurement of table pipeline,
we propose two test cases to obtain  the time  for evaluating the pipeline
processing performance. The proposed test case for the execution time
measurement of the action set is more accurately  than the method proposed by OFLOPS
\cite{OFLOPS}. Finally, we propose a method to verify the accuracy of idle and
hard timeout for the flow entry

\begin{center}
    \begin{table}[htb]
        \centering
        \begin{tabular}{| l | P{2cm} | l | P{2cm} | l | P{2cm} | P{2cm} |}
            \hline
            Type & Parameter & Spirent \cite{Spirent_whitepaper} & OFLOPS \cite{OFLOPS}
                 & Bianco \cite{Bianco2010} & Handfield \cite{SDN_Flow_Tables} & Ours \\ \hline
            D2C & Packet-in rate & Concept & N/A & N/A & N/A & Buffer size \\ \hline
            \multirow{2}{*}{C2D} & Packet-out rate & Concept & N/A & N/A & N/A & Buffer size \\ \cline{2-7}
                                 & Latency of Flow-mod & N/A & Traffic validation
                                    & N/A & Impact factors & N/A \\ \hline
            \multirow{5}{*}{OFD} & Table lookup & Concept & N/A & Constant & N/A & N/A\\ \cline{2-7}
                                 & Table pipeline & Concept & N/A & N/A & N/A & Time, pipeline gain
                                    \\ \cline{2-7}
                                 & Execution time of action set & N/A & End-to-end
                                    & N/A & N/A & Exactly action time \\ \cline{2-7}
                                 & The timeout of flow entry & Concept & N/A & N/A
                                    & N/A & Accuracy \\ \cline{2-7}
                                & The size of flow table & Concept & N/A & N/A & N/A & Trivial \\
            \hline
        \end{tabular}
        \caption{Related work comparison}
        \label{table:related_work}
    \end{table}
\end{center}
