我是一個使用 Spirit 的新手。
我正在嘗試使用精神 x3 從一個簡單的“excel”公式構建一個 AST 樹。語法支持典型的運算子( 、-、*、/)、函式(myfunc(myparam1, myparam2))和單元格參考(例如A1、AA234)。
因此,要決議的示例運算式可能是 A1 sin(A2 3)。
問題是下面的 xlreference 規則永遠不會匹配,因為 xlfunction 規則優先并且規則不會回溯。我已經嘗試過expect,但我缺乏一些很好的例子來讓它繼續下去。
我想這會引出另一個與除錯 x3 的最佳方法有關的問題。我已經看到了 BOOST_SPIRIT_X3_DEBUG 定義,但我找不到任何證明它的用法的例子。我還在 expression_class 上寫了一個 on_error 方法,但這并沒有提供很好的跟蹤。我曾嘗試使用位置標記和 with 陳述句,但這也沒有提供足夠的資訊。
任何幫助,將不勝感激!
x3::rule<class xlreference, ast::xlreference> const xlreference{"xlreference"};
auto const xlreference_def = alpha > x3::uint_ > !x3::expect[char('(')];
BOOST_SPIRIT_DEFINE(xlreference);
struct identifier_class;
typedef x3::rule<identifier_class, std::string> identifier_type;
identifier_type const identifier = "identifier";
auto const identifier_def = x3::char_("a-zA-Z") > *(x3::char_("a-zA-Z") | x3::char_('_')) > !x3::expect[char('0-9')];
BOOST_SPIRIT_DEFINE(identifier);
auto const expression_def = // constadditive_expr_def
term [print_action()]
>> *( (char_(' ') > term)
| (char_('-') > term)
)
;
x3::rule<xlfunction_class, ast::xlfunction> const xlfunction("xlfunction");
auto const xlfunction_def = identifier > '(' > *(expression > *(',' > expression)) > ')';
BOOST_SPIRIT_DEFINE(xlfunction);
auto const term_def = //constmultiplicative_expr_def
factor
>> *( (char_('*') > factor)
| (char_('/') > factor)
)
;
auto const factor_def = // constunary_expr_def
xlfunction [print_action()]
| '(' > expression > ')'
| (char_('-') > factor)
| (char_(' ') > factor)
| x3::double_ [print_action()] | xlreference [print_action()]
;
錯誤處理程式:
struct expression_class //: x3::annotate_on_success
{
// Our error handler
template <typename Iterator, typename Exception, typename Context>
x3::error_handler_result
on_error(Iterator& q, Iterator const& last, Exception const& x, Context const& context)
{
std::cout
<< "Error! Expecting: "
<< x.which()
<< " here: \""
<< std::string(x.where(), last)
<< "\""
<< std::endl
;
return x3::error_handler_result::fail;
}
};
位置標簽:
with<position_cache_tag>(std::ref(positions))
[
client::calculator_grammar::expression
];
client::ast::program ast;
bool r = phrase_parse(iter, (iterator_type const ) str.end(), parser, x3::space, ast);
if (!r) {
std::cout << "failed:" << str << "\n";
}
uj5u.com熱心網友回復:
好的,一步一步地進行審查。一路組成 AST 型別(因為你不想展示)。
這是無效代碼:
char('0-9')。那是寬字符文字嗎?啟用編譯器警告!您可能的意思是x3::char_("0-9")(兩個重要的區別!)。!x3::expect[]是矛盾的。您永遠無法通過該條件,因為!斷言前瞻不匹配,而expect[]需要匹配。因此,最好的情況是!失敗,因為expect[]-ation 是匹配的。最壞的情況是expect[]拋出例外,因為你要求它。operator >已經是一個期待點。由于與以前相同的原因> !p是矛盾的。做了>> !p替換
char_("0-9")為x3::digit替換
char_("a-zA-Z")為x3::alpha一些(許多)規則需要是詞素。那是因為您在船長背景關系(
phrase_parsewithx3::space)中呼叫了語法。你的識別符號會默默地吃空白,因為你沒有讓它們成為詞素。請參閱提升精神船長問題否定前瞻斷言不公開屬性,因此
! char_('(')可以(應該?)! lit('(')語意動作的存在(默認情況下)會抑制屬性傳播 - 因此
print_action()會導致屬性傳播停止operator>定義的期望點 ( ) 不能回溯。這就是他們期望值的原因。使用與 kleene-star 組合的 List 運算子:
p >> *(',' >> p)->p % ','那個額外的 kleene-star 是假的。您的意思是將引數串列設為可選嗎?那是
-(expression % ',')鏈接運算子規則使得獲取 ast 變得有點麻煩
簡化
factor >> *((x3::char_('*') > factor) // | (x3::char_('/') > factor));只是
factor >> *(x3::char_("*/") >> factor);factor_def邏輯上匹配什么expression?
第一次審查通過產生:
auto const xlreference_def =
x3::lexeme[ x3::alpha >> x3::uint_] >> !x3::char_('(');
auto const identifier_def =
x3::raw[x3::lexeme[x3::alpha >> *(x3::alpha | '_') >> !x3::digit]];
auto const xlfunction_def = identifier >> '(' >> -(expression % ',') >> ')';
auto const term_def = factor >> *(x3::char_("*/") >> factor);
auto const factor_def = xlfunction //
| '(' >> expression >> ')' //
| x3::double_ //
| xlreference;
auto const expression_def = term >> *(x3::char_("- ") >> term);
更多觀察:
(iterator_type const)str.end()?? 永遠不要使用 C 風格的強制轉換。事實上,只要使用str.cend()或事實上str.end(),如果str是適當的const反正。phrase_parse- 考慮不要讓船長成為來電者的決定,因為它在邏輯上是你語法的一部分多種excel運算式沒有決議:A:A、$A4、B$4、所有單元格區域;我認為通常也支持 R1C1
魔法仙塵的時間
因此,憑借豐富的經驗,我要去 Crystall Ball? 一些 AST?:
namespace client::ast {
using identifier = std::string;
//using identifier = boost::iterator_range<std::string::const_iterator>;
struct string_literal : std::string {
using std::string::string;
using std::string::operator=;
friend std::ostream& operator<<(std::ostream& os, string_literal const& sl) {
return os << std::quoted(sl) ;
}
};
struct xlreference {
std::string colname;
size_t rownum;
};
struct xlfunction; // fwd
struct binary_op; // fwd
using expression = boost::variant< //
double, //
string_literal, //
identifier, //
xlreference, //
boost::recursive_wrapper<xlfunction>, //
boost::recursive_wrapper<binary_op> //
>;
struct xlfunction{
identifier name;
std::vector<expression> args;
friend std::ostream& operator<<(std::ostream& os, xlfunction const& xlf)
{
os << xlf.name << "(";
char const* sep = "";
for (auto& arg : xlf.args)
os << std::exchange(sep, ", ") << arg;
return os;
}
};
struct binary_op {
struct chained_t {
char op;
expression e;
};
expression lhs;
std::vector<chained_t> chained;
friend std::ostream& operator<<(std::ostream& os, binary_op const& bop)
{
os << "(" << bop.lhs;
for (auto& rhs : bop.chained)
os << rhs.op << rhs.e;
return os << ")";
}
};
using program = expression;
using boost::fusion::operator<<;
} // namespace client::ast
我們迅速適應:
BOOST_FUSION_ADAPT_STRUCT(client::ast::xlfunction, name, args)
BOOST_FUSION_ADAPT_STRUCT(client::ast::xlreference, colname, rownum)
BOOST_FUSION_ADAPT_STRUCT(client::ast::binary_op, lhs, chained)
BOOST_FUSION_ADAPT_STRUCT(client::ast::binary_op::chained_t, op, e)
接下來,讓我們宣告合適的規則:
x3::rule<struct identifier_class, ast::identifier> const identifier{"identifier"};
x3::rule<struct xlreference, ast::xlreference> const xlreference{"xlreference"};
x3::rule<struct xlfunction_class, ast::xlfunction> const xlfunction{"xlfunction"};
x3::rule<struct factor_class, ast::expression> const factor{"factor"};
x3::rule<struct expression_class, ast::binary_op> const expression{"expression"};
x3::rule<struct term_class, ast::binary_op> const term{"term"};
其中需要定義:
auto const xlreference_def =
x3::lexeme[ x3::alpha >> x3::uint_] /*>> !x3::char_('(')*/;
請注意,前瞻斷言 (
!) 實際上并未更改決議結果,因為任何單元格參考無論如何都不是有效識別符號,因此 () 將保持未決議狀態。
auto const identifier_def =
x3::raw[x3::lexeme[x3::alpha >> *(x3::alpha | '_') /*>> !x3::digit*/]];
同樣在這里。剩余的輸入將在
x3::eoi稍后進行檢查。
我輸入了一個字串文字,因為任何 Excel 克隆都會有一個:
auto const string_literal =
x3::rule<struct _, ast::string_literal>{"string_literal"} //
= x3::lexeme['"' > *('\\' >> x3::char_ | ~x3::char_('"')) > '"'];
Note that this demonstrates that non-recursive, locally-defined rules don't need separate definitions.
Then come the expression rules
auto const factor_def = //
xlfunction //
| '(' >> expression >> ')' //
| x3::double_ //
| string_literal //
| xlreference //
| identifier //
;
I'd usually call this "simple expression" instead of factor.
auto const term_def = factor >> *(x3::char_("*/") >> factor);
auto const expression_def = term >> *(x3::char_("- ") >> term);
auto const xlfunction_def = identifier >> '(' >> -(expression % ',') >> ')';
Straight-forward mappings to the AST.
BOOST_SPIRIT_DEFINE(xlreference)
BOOST_SPIRIT_DEFINE(identifier)
BOOST_SPIRIT_DEFINE(xlfunction)
BOOST_SPIRIT_DEFINE(term)
BOOST_SPIRIT_DEFINE(factor)
BOOST_SPIRIT_DEFINE(expression)
'Nuff said. Now comes a bit of cargo cult - remnants of code both unshown and unused, which I'll mostly just accept and ignore here:
int main() {
std::vector<int> positions; // TODO
auto parser = x3::with<struct position_cache_tag /*TODO*/> //
(std::ref(positions)) //
[ //
x3::skip(x3::space)[ //
client::calculator_grammar::expression >> x3::eoi //
] //
];
DO NOTE though that
x3::eoimakes it so the rule doesn't match if end of input (modulo skipper) isn't reached.
Now, let's add some test cases!
struct {
std::string category;
std::vector<std::string> cases;
} test_table[] = {
{
"xlreference",
{"A1", "A1111", "AbCdZ9876543", "i9", "i0"},
},
{
"identifier",
{"i", "id", "id_entifier"},
},
{
"number",
{"123", "inf", "-inf", "NaN", ".99e34", "1e-8", "1.e-8", " 9"},
},
{
"binaries",
{ //
"3 4", "3*4", //
"3 4 5", "3*4*5", "3 4*5", "3*4 5", "3*4 5", //
"3 (4 5)", "3*(4*5)", "3 (4*5)", "3*(4 5)", "3*(4 5)", //
"(3 4) 5", "(3*4)*5", "(3 4)*5", "(3*4) 5", "(3*4) 5"},
},
{
"xlfunction",
{
"pi()",
"sin(4)",
R"--(IIF(A1, "Red", "Green"))--",
},
},
{
"invalid",
{
"A9()", // an xlreference may not be followed by ()
"", // you didn't specify
},
},
{
"other",
{
"A-9", // 1-letter identifier and binary operation
"1 9", // unary plus accepted in number rule
},
},
{
"question",
{
"myfunc(myparam1, myparam2)",
"A1",
"AA234",
"A1 sin(A2 3)",
},
},
};
And run them:
for (auto& [cat, cases] : test_table) {
for (std::string const& str : cases) {
auto iter = begin(str), last(end(str));
std::cout << std::setw(12) << cat << ": ";
client::ast::program ast;
if (parse(iter, last, parser, ast)) {
std::cout << "parsed: " << ast;
} else {
std::cout << "failed: " << std::quoted(str);
}
if (iter == last) {
std::cout << "\n";
} else {
std::cout << " unparsed: "
<< std::quoted(std::string_view(iter, last)) << "\n";
}
}
}
Live Demo
Live On Coliru
//#define BOOST_SPIRIT_X3_DEBUG
#include <boost/fusion/adapted.hpp>
#include <boost/fusion/include/io.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/position_tagged.hpp>
#include <boost/spirit/home/x3/support/utility/annotate_on_success.hpp>
#include <iostream>
#include <iomanip>
#include <map>
namespace x3 = boost::spirit::x3;
namespace client::ast {
using identifier = std::string;
//using identifier = boost::iterator_range<std::string::const_iterator>;
struct string_literal : std::string {
using std::string::string;
using std::string::operator=;
friend std::ostream& operator<<(std::ostream& os, string_literal const& sl) {
return os << std::quoted(sl) ;
}
};
struct xlreference {
std::string colname;
size_t rownum;
};
struct xlfunction; // fwd
struct binary_op; // fwd
using expression = boost::variant< //
double, //
string_literal, //
identifier, //
xlreference, //
boost::recursive_wrapper<xlfunction>, //
boost::recursive_wrapper<binary_op> //
>;
struct xlfunction{
identifier name;
std::vector<expression> args;
friend std::ostream& operator<<(std::ostream& os, xlfunction const& xlf)
{
os << xlf.name << "(";
char const* sep = "";
for (auto& arg : xlf.args)
os << std::exchange(sep, ", ") << arg;
return os;
}
};
struct binary_op {
struct chained_t {
char op;
expression e;
};
expression lhs;
std::vector<chained_t> chained;
friend std::ostream& operator<<(std::ostream& os, binary_op const& bop)
{
os << "(" << bop.lhs;
for (auto& rhs : bop.chained)
os << rhs.op << rhs.e;
return os << ")";
}
};
using program = expression;
using boost::fusion::operator<<;
} // namespace client::ast
BOOST_FUSION_ADAPT_STRUCT(client::ast::xlfunction, name, args)
BOOST_FUSION_ADAPT_STRUCT(client::ast::xlreference, colname, rownum)
BOOST_FUSION_ADAPT_STRUCT(client::ast::binary_op, lhs, chained)
BOOST_FUSION_ADAPT_STRUCT(client::ast::binary_op::chained_t, op, e)
namespace client::calculator_grammar {
struct expression_class //: x3::annotate_on_success
{
// Our error handler
template <typename Iterator, typename Exception, typename Context>
x3::error_handler_result on_error(Iterator& q, Iterator const& last,
Exception const& x,
Context const& context)
{
std::cout //
<< "Error! Expecting: " << x.which() //
<< " here: \"" << std::string(x.where(), last) //
<< "\"" << std::endl;
return x3::error_handler_result::fail;
}
};
x3::rule<struct identifier_class, ast::identifier> const identifier{"identifier"};
x3::rule<struct xlreference, ast::xlreference> const xlreference{"xlreference"};
x3::rule<struct xlfunction_class, ast::xlfunction> const xlfunction{"xlfunction"};
x3::rule<struct factor_class, ast::expression> const factor{"factor"};
x3::rule<struct expression_class, ast::binary_op> const expression{"expression"};
x3::rule<struct term_class, ast::binary_op> const term{"term"};
auto const xlreference_def =
x3::lexeme[ x3::alpha >> x3::uint_] /*>> !x3::char_('(')*/;
auto const identifier_def =
x3::raw[x3::lexeme[x3::alpha >> *(x3::alpha | '_') /*>> !x3::digit*/]];
auto const string_literal =
x3::rule<struct _, ast::string_literal>{"string_literal"} //
= x3::lexeme['"' > *('\\' >> x3::char_ | ~x3::char_('"')) > '"'];
auto const factor_def = //
xlfunction //
| '(' >> expression >> ')' //
| x3::double_ //
| string_literal //
| xlreference //
| identifier //
;
auto const term_def = factor >> *(x3::char_("*/") >> factor);
auto const expression_def = term >> *(x3::char_("- ") >> term);
auto const xlfunction_def = identifier >> '(' >> -(expression % ',') >> ')';
BOOST_SPIRIT_DEFINE(xlreference)
BOOST_SPIRIT_DEFINE(identifier)
BOOST_SPIRIT_DEFINE(xlfunction)
BOOST_SPIRIT_DEFINE(term)
BOOST_SPIRIT_DEFINE(factor)
BOOST_SPIRIT_DEFINE(expression)
} // namespace client::calculator_grammar
int main() {
std::vector<int> positions; // TODO
auto parser = x3::with<struct position_cache_tag /*TODO*/> //
(std::ref(positions)) //
[ //
x3::skip(x3::space)[ //
client::calculator_grammar::expression >> x3::eoi //
] //
];
struct {
std::string category;
std::vector<std::string> cases;
} test_table[] = {
{
"xlreference",
{"A1", "A1111", "AbCdZ9876543", "i9", "i0"},
},
{
"identifier",
{"i", "id", "id_entifier"},
},
{
"number",
{"123", "inf", "-inf", "NaN", ".99e34", "1e-8", "1.e-8", " 9"},
},
{
"binaries",
{ //
"3 4", "3*4", //
"3 4 5", "3*4*5", "3 4*5", "3*4 5", "3*4 5", //
"3 (4 5)", "3*(4*5)", "3 (4*5)", "3*(4 5)", "3*(4 5)", //
"(3 4) 5", "(3*4)*5", "(3 4)*5", "(3*4) 5", "(3*4) 5"},
},
{
"xlfunction",
{
"pi()",
"sin(4)",
R"--(IIF(A1, "Red", "Green"))--",
},
},
{
"invalid",
{
"A9()", // an xlreference may not be followed by ()
"", // you didn't specify
},
},
{
"other",
{
"A-9", // 1-letter identifier and binary operation
"1 9", // unary plus accepted in number rule
},
},
{
"question",
{
"myfunc(myparam1, myparam2)",
"A1",
"AA234",
"A1 sin(A2 3)",
},
},
};
for (auto& [cat, cases] : test_table) {
for (std::string const& str : cases) {
auto iter = begin(str), last(end(str));
std::cout << std::setw(12) << cat << ": ";
client::ast::program ast;
if (parse(iter, last, parser, ast)) {
std::cout << "parsed: " << ast;
} else {
std::cout << "failed: " << std::quoted(str);
}
if (iter == last) {
std::cout << "\n";
} else {
std::cout << " unparsed: "
<< std::quoted(std::string_view(iter, last)) << "\n";
}
}
}
}
Prints
xlreference: parsed: (((A 1)))
xlreference: parsed: (((A 1111)))
xlreference: parsed: (((AbCdZ 9876543)))
xlreference: parsed: (((i 9)))
xlreference: parsed: (((i 0)))
identifier: parsed: ((i))
identifier: parsed: ((id))
identifier: parsed: ((id_entifier))
number: parsed: ((123))
number: parsed: ((inf))
number: parsed: ((-inf))
number: parsed: ((nan))
number: parsed: ((9.9e 33))
number: parsed: ((1e-08))
number: parsed: ((1e-08))
number: parsed: ((9))
binaries: parsed: ((3) (4))
binaries: parsed: ((3*4))
binaries: parsed: ((3) (4) (5))
binaries: parsed: ((3*4*5))
binaries: parsed: ((3) (4*5))
binaries: parsed: ((3*4) (5))
binaries: parsed: ((3*4) (5))
binaries: parsed: ((3) (((4) (5))))
binaries: parsed: ((3*((4*5))))
binaries: parsed: ((3) (((4*5))))
binaries: parsed: ((3*((4) (5))))
binaries: parsed: ((3*((4) (5))))
binaries: parsed: ((((3) (4))) (5))
binaries: parsed: ((((3*4))*5))
binaries: parsed: ((((3) (4))*5))
binaries: parsed: ((((3*4))) (5))
binaries: parsed: ((((3*4))) (5))
xlfunction: parsed: ((pi())
xlfunction: parsed: ((sin(((4))))
xlfunction: parsed: ((IIF((((A 1))), (("Red")), (("Green"))))
invalid: failed: "A9()" unparsed: "A9()"
invalid: failed: ""
other: parsed: ((A)-(9))
other: parsed: ((1) (9))
question: parsed: ((myfunc((((myparam 1))), (((myparam 2)))))
question: parsed: (((A 1)))
question: parsed: (((AA 234)))
question: parsed: (((A 1)) (sin((((A 2)) (3))))
The only two failed lines are as expected
DEBUG?
Simply uncomment
#define BOOST_SPIRIT_X3_DEBUG
And be slammed with additional noise:
question: <expression>
<try>A1 sin(A2 3)</try>
<term>
<try>A1 sin(A2 3)</try>
<factor>
<try>A1 sin(A2 3)</try>
<xlfunction>
<try>A1 sin(A2 3)</try>
<identifier>
<try>A1 sin(A2 3)</try>
<success>1 sin(A2 3)</success>
<attributes>[A]</attributes>
</identifier>
<fail/>
</xlfunction>
<string_literal>
<try>A1 sin(A2 3)</try>
<fail/>
</string_literal>
<xlreference>
<try>A1 sin(A2 3)</try>
<success> sin(A2 3)</success>
<attributes>[[A], 1]</attributes>
</xlreference>
<success> sin(A2 3)</success>
<attributes>[[A], 1]</attributes>
</factor>
<success> sin(A2 3)</success>
<attributes>[[[A], 1], []]</attributes>
</term>
<term>
<try> sin(A2 3)</try>
<factor>
<try> sin(A2 3)</try>
<xlfunction>
<try> sin(A2 3)</try>
<identifier>
<try> sin(A2 3)</try>
<success>(A2 3)</success>
<attributes>[s, i, n]</attributes>
</identifier>
<expression>
<try>A2 3)</try>
<term>
<try>A2 3)</try>
<factor>
<try>A2 3)</try>
<xlfunction>
<try>A2 3)</try>
<identifier>
<try>A2 3)</try>
<success>2 3)</success>
<attributes>[A]</attributes>
</identifier>
<fail/>
</xlfunction>
<string_literal>
<try>A2 3)</try>
<fail/>
</string_literal>
<xlreference>
<try>A2 3)</try>
<success> 3)</success>
<attributes>[[A], 2]</attributes>
</xlreference>
<success> 3)</success>
<attributes>[[A], 2]</attributes>
</factor>
<success> 3)</success>
<attributes>[[[A], 2], []]</attributes>
</term>
<term>
<try>3)</try>
<factor>
<try>3)</try>
<xlfunction>
<try>3)</try>
<identifier>
<try>3)</try>
<fail/>
</identifier>
<fail/>
</xlfunction>
<success>)</success>
<attributes>3</attributes>
</factor>
<success>)</success>
<attributes>[3, []]</attributes>
</term>
<success>)</success>
<attributes>[[[[A], 2], []], [[ , [3, []]]]]</attributes>
</expression>
<success></success>
<attributes>[[s, i, n], [[[[[A], 2], []], [[ , [3, []]]]]]]</attributes>
</xlfunction>
<success></success>
<attributes>[[s, i, n], [[[[[A], 2], []], [[ , [3, []]]]]]]</attributes>
</factor>
<success></success>
<attributes>[[[s, i, n], [[[[[A], 2], []], [[ , [3, []]]]]]], []]</attributes>
</term>
<success></success>
<attributes>[[[[A], 1], []], [[ , [[[s, i, n], [[[[[A], 2], []], [[ , [3, []]]]]]], []]]]]</attributes>
</expression>
parsed: (((A 1)) (sin((((A 2)) (3))))
轉載請註明出處,本文鏈接:https://www.uj5u.com/gongcheng/330410.html
下一篇:簡化嵌套的for回圈
