Screening of small molecule inhibitors against RhoA protein from Alisma using an online detection system

•The antitumor activity of Rhizoma alismatis was discovered from an UPLC-PDA-ESI-Q-TOF-MS2-FLD protein activity screening system.•A total of 17 active compounds from Rhizoma alismatis were separated and identified.•The identified compounds play tumor suppressive roles through inhibiting tumor proliferation, invasion and metastasis.•Expressions of the RhoA-associated proteins were detected to be significantly downregulated by the 17 active compounds.