The Euclidean space is the familiar space for training neural models and performing arithmetic operations. However, many data types inherently possess complex geometries, and model training methods involve operating over their latent representations, which cannot be effectively captured in the Euclidean space. The hyperbolic space provides a more generalized representative geometry to model the hierarchical complexities of the tree-like structure of natural language. We propose \textscAdaPT a set of guidelines for initialization, parametrization, and training of neural networks, which adapts to the dataset and can be used with different manifolds. \textscAdaPT can be generalized over \textitany existing neural network training methodology and leads to more stable training without a substantial increase in training time. We apply \textscAdaPT guidelines over two state-of-the-art deep learning approaches and empirically demonstrate its effectiveness through experiments on three tasks over 12 languages across speech and text. Through extensive qualitative analysis, we put forward the applicability of \textscAdaPT as a set of guidelines optimally utilizing the manifold geometry, which can be extended to various downstream tasks across languages and modalities.

@inproceedings{Sawhney-et-al-emnlp-22, abstract = {The Euclidean space is the familiar space for training neural models and performing arithmetic operations. However, many data types inherently possess complex geometries, and model training methods involve operating over their latent representations, which cannot be effectively captured in the Euclidean space. The hyperbolic space provides a more generalized representative geometry to model the hierarchical complexities of the tree-like structure of natural language. We propose \textsc{AdaPT} a set of guidelines for initialization, parametrization, and training of neural networks, which adapts to the dataset and can be used with different manifolds. \textsc{AdaPT} can be generalized over \textit{any} existing neural network training methodology and leads to more stable training without a substantial increase in training time. We apply \textsc{AdaPT} guidelines over two state-of-the-art deep learning approaches and empirically demonstrate its effectiveness through experiments on three tasks over 12 languages across speech and text. Through extensive qualitative analysis, we put forward the applicability of \textsc{AdaPT} as a set of guidelines optimally utilizing the manifold geometry, which can be extended to various downstream tasks across languages and modalities.}, address = {Abu Dhabi, UAE}, author = {Ramit Sawhney and Megh Thakkar and Vishwa Shah and Shrey Pandit and Shafiq Joty}, booktitle = {the 2022 Conference on Empirical Methods in Natural Language Processing}, publisher = {ACL}, series = {EMNLP'22}, title = {AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP}, url = {}, year = {2022} }