HPC system procurement with a fixed budget is an optimization problem with many trade-offs. In particular, the choice of an interconnection network for a system is a major choice, since communication performance is important to overall application performance and network makes up a substantial fraction of a supercomputerÕs overall price. It is necessary to understand how sensitive representative jobs are to various aspects of network performance to procure the right network. Previous studies used mostly communication-only motifs or simulation; in this study, we use dedicated time on a cluster at Lawrence Livermore National Lab to measure the performance of application running in various controlled environments. We vary background congestion, mapping and placement, and observe the impact on overall application performance. Overall, we find that a 2:1 tapered fat tree provides sufficiently robust communication performance for a representative mix of applications while generating meaningful cost savings relative to a full bisection bandwidth fat tree. Furthermore, our results advise against further tapering, as the resulting performance degradation would exceed cost savings. However, application-specific mappings and topology-aware schedulers may reduce global bandwidth needs, providing room for additional network tapering.