Despite the increased popularity of online tools for remote teamwork and meetings, moderated collaborative activities between multiple users in early conceptual design stages, such as brainstorming sessions, are yet not well supported. In this paper, we introduce All4One, a networked system that enables multiple remote users to participate in a moderated visual sketching session. Each participant can independently draw and share sketches using a tablet, and a moderator uses a set of tangible tools to arrange and manipulate sketches that are displayed in real-time on a whiteboard. We present our prototype in detail and the results from a workshop study simulating a brainstorming session with designers who tested the system in practice. Results show several usage patterns and the potential of All4One for use in early design stages, and the importance of the role of the moderator as the facilitator of the design process. The paper concludes by identifying weaknesses and strengths of the current system and possible directions for future work.