Single-walled carbon nanotubes (SWCNTs) exhibit sharp excitonic resonances in their near-infrared optical spectra even when aggregated in macroscale assemblies. This characteristic offers promising opportunities for utilizing SWCNT assemblies as excitonic optofunctional materials. However, as a crucial factor determining the dominance of excitonic resonance, exciton binding energy in high-purity structure-sorted SWCNT membranes remains to be clarified. Here, we report modifications to the exciton binding energy in high-purity, structure-sorted semiconducting SWCNTs due to aggregation. We prepared membranes consisting of aggregated SWCNTs with high single chiral structure purity. Using one and two-photon spectroscopies, we observed the shrinkage in the energy spacing of the exciton Rydberg series in aggregated SWCNTs. By comparison of the experimental results with theoretical calculations, the exciton binding energy of the aggregated SWCNT membrane was deduced to be about 0.26 eV, maintaining approximately 80% of the value in the isolated SWCNTs stabilized by wrapping with organic polymers in toluene. These findings show promising prospects for utilizing exciton resonances in SWCNT aggregations not only at low temperatures but also at high temperatures.