The immune system protects the body against cancer by recognizing and eliminating neoplastic cells. Immune effector cells physically interact with cancer cells through a specialized membrane interface known as the immunological synapse. Such intimate interaction is necessary for immune cell activation and directional killing of target cells. A vast array of studies has established that actin cytoskeleton remodeling and membrane/vesicle trafficking play multiple and critical roles on the immune cell side of the immunological synapse. Increasing evidence supports that dynamic changes in actin cytoskeleton organization and vesicle trafficking also take place on the cancer cell side of the immunological synapse and that such changes are closely intertwined with evasion from immune destruction. In the present article, we review current knowledge of actin and vesicle dynamics in cancer cells during cytotoxic lymphocyte attack and tumor immune evasion.