Today with the growing needs of power, one of the goals of the HPC community is
to build larger systems given a strict power budget. The goal is not only to
build larger systems but also to optimize the systems' performance under the
power budget constraints. It is noted that running systems at their TDP
\footnote{ The thermal design power (TDP), sometimes called thermal design
  point, refers to the maximum amount of heat generated by the CPU, which the
    cooling system in a computer is required to dissipate in typical
    operation.} is a huge wastage since most of the times they are consuming
    lesser power than their TDP. Intel's Running Average Power Limit\cite{rapl}
    toolkit is a feature that helps to constrain the power of its compute cores
    and DRAM and thereby enabling software controlled, optimized power
    allocation to the compute nodes based on the application running on them.
    It is noted that under a strict power budget and under certain
    circumstances, running applications with lower power limits on more number
    of nodes can be more efficient than running the same application with higher
    power on fewer nodes \cite{powerCluster2013}.


The motivation for this paper is drawn from the future guidelines of the work
by Osman et al.\cite{powerCluster2013} and \cite{osmanthesis}.  It has been
empirically observed that under the same power cap, different nodes yield
different application performance. This can be due to several design factors:
difference in chip designs, different in component assembly by the machine
vendor, location of the node in the data center, difference in component design
such as fans, etc. This difference in the design causes load imbalance across
nodes despite same allocated power and equal compute load. Moreover we are
going to experimentally show that this heterogeneity across the nodes is more
prominent at lower power caps.  Our goal is to minimize the load imbalance in
the presence of such heterogeneity among the nodes using the over-decomposition
and dynamic object migration features of Charm++\cite{ChareKernelICPP90}.

Our work is a two-fold approach. Firstly we study the extent of heterogeneity
under the lower power caps and based on this study we design and implement a
power-aware load-balancer which amortizes this heterogeneity.

The rest of the paper is organized as follows. Section~\ref{sec:heterstudy}
talks about the applications and testbed that we used to explore the
heterogeneity at lower power caps. 
  %%study and the motivation for designing a power aware
  Section~\ref{sec:design} describes the design and implementation of our power
  aware load balancer.  Section~\ref{sec:results} discusses the performance of
  the power aware load balancer. Finally the Section~\ref{sec:fw} points out
  the conclusion and the future work.
