Social Workers (AI Conversation Evaluation)

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview We are looking for experienced professionals in psychology, social work, counseling, communication, qualitative research, HCI, conflict resolution, or related advisory disciplines to support an AI conversation-evaluation project. The role involves reviewing conversations between people and AI systems, assessing whether the AI gathered enough context before offering recommendations, identifying important missing information, and proposing relevant follow-up questions. Reviewers will distinguish between critical context and non-essential details and provide structured feedback to improve the quality of AI-assisted conversations.

CONTRACT: Contractor assignment, 9 weeks COMMITMENT: Full-time, 40 hours per week with at least 4 hours of PST overlap LOCATIONS: Remote - Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Pakistan, Indonesia, Turkey, or Vietnam PROCESS: short online test and delivery review

Key Responsibilities • Review AI conversations involving interpersonal, advisory, and everyday decision-making scenarios. • Assess whether the AI gathered sufficient context before making recommendations. • Identify important missing information and suggest relevant follow-up questions. • Determine whether additional context could materially change the recommendation. • Distinguish between load-bearing information and non-essential details. • Conduct structured qualitative reviews using defined evaluation guidelines. • Document findings clearly in spreadsheets, annotation tools, or labeling platforms. • Apply consistent judgment across cases and flag ambiguous examples. • Collaborate with research and quality teams to improve evaluation standards. • Participate in calibration and quality-review activities.