A Poincaré–Bendixson theorem for translation lines and applications to prime ends

Abstract

For an orientation-preserving homeomorphism of the sphere, we prove that if a translation line does not accumulate in a fixed point, then it necessarily spirals towards a topological attractor. This is in analogy with the description of flow lines given by Poincaré–Bendixson theorem. We then apply this result to the study of invariant continua without fixed points, in particular to circloids and boundaries of simply connected open sets. Among the applications, we show that if the prime ends rotation number of such an open set $U$ vanishes, then either there is a fixed point in the boundary, or the boundary of $U$ is contained in the basin of a finite family of topological “rotational” attractors. This description strongly improves a previous result by Cartwright and Littlewood, by passing from the prime ends compactification to the ambient space. Moreover, the dynamics in a neighborhood of the boundary is semiconjugate to a very simple model dynamics on a planar graph. Other applications involve the decomposability of invariant continua, and realization of rotation numbers by periodic points on circloids.