The syntheses of amphiphilic block copolymers are successfully performed in water by chain extension of hydrophilic macromolecules with styrene at 80 °C. The employed strategy is a one‐pot procedure in which poly(acrylic acid), poly(methacrylic acid) or poly(methacrylic acid‐co‐poly(ethylene oxide) methyl ether methacrylate) macroRAFTs are first formed in water using 4‐cyano‐4‐thiothiopropylsulfanyl pentanoic acid (CTPPA) as a chain transfer agent. The resulting macroRAFTs are then directly used without further purification for the RAFT polymerization of styrene in water in the same reactor. This simple and straightforward strategy leads to a very good control of the resulting amphiphilic block copolymers.