In recent years, language and multimedia models have made significant advancements, achieving remarkable performance on a large variety of tasks, including question answering, summarization, scientific reasoning, procedural understanding, and action planning, along with demonstrating robust zero-shot/few-shot capabilities, bolstered by model scaling and innovative training techniques. Despite the exciting progress, ensuring these models align with fundamental constitutional principles remains a critical challenge. In this talk, we present a roadmap for foundation models as digital agents that are not only technologically advanced and functionally robust, but also sociocultural-aware and ethically grounded. We begin by delving into advanced mechanisms (i.e., the InfoSurgeon architecture) for identifying information inconsistencies in textual or multimedia content, along with novel strategies to counter the undesirable phenomena of generative model hallucination. Moreover, we introduce norm discovery with self-verification on-the-fly (i.e., the NormSAGE framework) as a promising solution for the explainable detection of real-world norm violation occurrences and for guiding harmless language model response generations. Particularly, we emphasize the importance of our investigative efforts in massively multicultural knowledge acquisition as a vital component to enrich model understanding of norms across diverse societal groups, ensuring more accurate and respectful human-centered interactions. By addressing these research problems and opportunities, we can help reinforce the relevance and responsibility of these foundational language and multimedia AI models in promoting healthy information communication in our increasingly interconnected world.