Proteins are dynamic molecules and often undergo conformational change upon ligand binding. It is widely accepted that flexible loop regions have a critical functional role in enzymes. Lack of consideration of binding site flexibility has led to failures in predicting protein functions and in successfully docking ligands with protein receptors. Here we address the question: which sequence and structural features distinguish the structurally flexible and rigid binding sites? We analyze high-resolution crystal structures of ligand bound (holo) and free (apo) forms of 41 proteins where no conformational change takes place upon ligand binding, 35 examples with moderate conformational change, and 22 cases where a large conformational change has been observed. We find that the number of residue-residue contacts observed per-residue (contact density) does not distinguish flexible and rigid binding sites, suggesting a role for specific interactions and amino acids in modulating the conformational changes. Examination of hydrogen bonding and hydrophobic interactions reveals that cases that do not undergo conformational change have high polar interactions constituting the binding pockets. Intriguingly, the large, aromatic amino acid tryptophan has a high propensity to occur at the binding sites of examples where a large conformational change has been noted. Further, in large conformational change examples, hydrophobic-hydrophobic, aromatic-aromatic, and hydrophobic-polar residue pair interactions are dominant. Further analysis of the Ramachandran dihedral angles (phi, psi) reveals that the residues adopting disallowed conformations are found in both rigid and flexible cases. More importantly, the binding site residues adopting disallowed conformations clustered narrowly into two specific regions of the L-Ala Ramachandran map. Examination of the dihedral angles changes upon ligand binding shows that the magnitude of phi, psi changes are in general minimal, although some large changes particularly between right-handed alpha-helical and extended conformations are seen. Our work further provides an account of conformational changes in the dihedral angles space. The findings reported here are expected to assist in providing a framework for predicting protein-ligand complexes and for template-based prediction of protein function.