10/21/2019 CSCI 561 (139 unread)
note
472 views
Proposed Spoiling Solution for HW2
UPDATE 2:
At this stage of the discussion, please focus on ways the rules change could be undermined and not how the changes feel to you. Thank you! UPDATES:
Clarified rule 1a.
Got rid of 1b since it’s moot (my bad). The former 1c is now 1b.
In order to address the additional opportunities for spoiling the game brought up in @210, we are planning to amend the rules of the game so wall building becomes impractical. While we think that these changes should address the remains avenues for spoiling the game, we are soliciting your input. Please discuss the proposed amendments in this thread.
=== Rules Changes ===
1. Moving pieces
a) Players cannot make a move that starts outside their own camp and causes one of their pieces to end up in their own camp. b) If a player has at least one piece left in their own camp, they have to
Move a piece out of their camp (i.e. at the end of the whole move the piece ends up outside of their camp).
If that¡¯s not possible, move a piece in their camp further away from the corner of their own camp ([0,0] or [15,15] respectively).
Only if the player does not have any pieces left in their camp or none of the two alternatives above are possible are they free to move pieces outside of their camp.
2. Winning the game (clarification)
A player wins if they are the first to fill out all the space in the opposite camp that¡¯s not occupied by the opposite player¡¯s pieces using at least one of their pieces. === End of Rules Changes ===
We¡¯re looking forward to your feedback!
Updated 12 days ago by Daniel Link
followup discussions for lingering questions and comments
Resolved
Unresolved
Anonymous 13 days ago
Changing rules midway is confusing. But this is welcome since it clarifies jump sequences.
So if these rules are implemented, i am to assume that I can only move my piece towards enemy camp and never backwards to my own camp? Example 2 and 3 in pdf has moves that makes it go back to its own camp. So this is invalid according to these rules right?
https://piazza.com/class/jzob0qt1owq15p?cid=334
1/17
2wh
10/21/2019 CSCI 561 (139 unread)
Can we get a new set of test cases?
Anonymous 12 days ago Clarifying the following rule:
“a) Players cannot make a move that causes one of their pieces to end up in their own camp.”
This only means we can’t put a piece back into our camp and not that we can’t move a piece towards our camp right?
Daniel Link 12 days ago If you start outside the camp and end up outside the camp, you can move back. We would change the examples.
Anonymous 11 days ago Please provide the updated assignment document with examples
Resolved
Unresolved
Anonymous 13 days ago For 1. b) , How will it be ever possible to jump over our own pieces if we can never be adjacent to our own pieces. Daniel Link 13 days ago Good consideration. This rule may have to be modified.
Daniel Link 12 days ago The rule has been modified :).
Resolved
Unresolved
Atharva Khare 13 days ago
Isn’t rule 1.b too radical to be changed 1.5 weeks into the homework? There are strategies (not involving spoiling) that depend on the currently legal moves. Applying 1b eliminates most jumps, making this a completely different game.
Can you clarify rule 1.c, maybe with examples?
“Only if the player does not have any pieces left in their camp or none of the two alternatives above are possible are they free to move pieces outside of their camp” > Does this mean that the AI has to compulsorily vacate home camp first, and then strategize how to reach enemy camp?
Daniel Link 12 days ago We’ll think about the jumps.
For 1c, what kind of examples would you be looking for? I can’t comment on how to strategize. Daniel Link 12 days ago The old 1b has been deleted.
Atharva Khare 12 days ago Since the old 1.b has been removed, it clarifies the new rule 1.b, so I won’t be needing any examples anymore 🙂
https://piazza.com/class/jzob0qt1owq15p?cid=334
2/17
10/21/2019 CSCI 561 (139 unread)
Resolved
Unresolved
Daniel Link 12 days ago Thank you!
yujie sun 12 days ago
I think rule 1.c is enough, 1.b is not needed. If you have a fortress, you have to move back and forth, which means you can not satisfy the “further away” condition.
Daniel Link 12 days ago The issue is that a player could build a wall outside their camp by filling out two diagonals with 7 and 8 (or 8 and 9) pieces and moving the rest around the board. 1b is to prevent that.
Daniel Link 12 days ago Never mind what I said one post higher about the wall. Check out the updates.
Resolved
Unresolved
Anonymous 12 days ago I think I have found a simple rule to avoid spoiling strategy. Rule:
1. Once a piece is out of its own camp, it cannot re-enter it¡¯s own camp.
2. At every 3 moves, total number of pieces in player¡¯s own camp must ATLEAST reduce by 1.
eg. after move 9, total pieces in camp should be <= 16 (19-3)
eg. after move 5, total number of pieces in camp should be <= 18 (19-1)
This will ensure that the player must always remove atleast 1 piece every 3 moves otherwise the player is considered LOST. This will guarantee to prevent spoiling as the number of pieces of player in its own camp must always decrease by ATLEAST 1 every 3 moves.
The new rules you mentioned are complex to implement and also kills the actual rules and strategy of the game. Therefore, please consider the rules I have suggested and please find any fault or provide suggestion if any.
yujie sun 12 days ago We do not know which round we are at, we only know the remaining time. How can we detect if this is move 5 or move 9?
Anonymous 12 days ago implement this.
Daniel Link 12 days ago Daniel Link 12 days ago
The TA¡¯s game agent should provide the extra input of move number along with the given inputs. So, it will become easy task for us to
A player could still build a wall outside their camp, see above.
Never mind what I said about the wall. I think we can do without counting moves.
https://piazza.com/class/jzob0qt1owq15p?cid=334
3/17
10/21/2019 CSCI 561 (139 unread)
Resolved
Unresolved
Botong Lyu 12 days ago
I think rule b needs to be abolished because it bans all hopping moves on friendly pieces completely. Without that move, a lot of key strategies in the game will be invalid. This rule take out the only unique feature in Halma. I think instead of modifying which move is allowed, we could modify the win condition so that certain draws no longer exists. For example, we could count how many sots of the a base is occupied when one of the agents runs out of time. And the base with less occupied slots wins. This also does not clash with wining when the opponents base is fully occupied and will eliminates most if not all stalling strategies.
In addition, no matter what the final rule is, I think we should be informed about it asap because it fundamentally changes our approach to the game's eval function (for instance, we cannot recognize certain established advantage patterns in Halma if some moves are not legal any more) and our finished work could be ruined. If a rule change, like this one, is necessary, I hope the deadline of this assignment could reflect that change to allow us finish the assignment properly. Personally, without a robust final rule, I simply cannot continue working on this assignment knowing that all of my work could be obsolete in the future.
Anonymous 12 days ago Agreed. And move generation also will be impacted because of rule change
Daniel Link 12 days ago Dealing only with the camps is not enough because a player could still build a wall outside their camp, see above. We can't make changes that don't affect any code, sorry.
Botong Lyu 12 days ago A diagonal wall can still be penetrated by jumping diagonally. I don't see a way of building a wall using only 19 pieces to block all penetrating movements.
I understand that changes will affect code. But a change this drastic from the original game means we cannot use established knowledge of the game to implement our agent. A minimax agent's performance relies heavily on the eval function. Without any prior knowledge on certain states being especially advantages or disadvantages means that we need to figure them out ourselves which makes the assignment more time consuming and the performance of our agent irrelevant to the core material of our course.
Daniel Link 12 days ago further comments.
Daniel Link 12 days ago
Unresolved
You're right about the wall. I made a mistake by doing this too late in the evening :). I'll update the rules and am looking forward to your I got rid of the previous 1b.
Resolved
Sourish Banerjee 12 days ago
I think 1a is also ambiguously worded because it never allows the second condition in 1c. If the player has a piece in his/her own camp that he/she cannot move out of the camp, then any move with this piece results in it still being in the home camp, which is disallowed by 1a.
I'm guessing 1a only applies to moves with pieces already outside the home camp, but this needs to be explicitly mentioned.
https://piazza.com/class/jzob0qt1owq15p?cid=334
4/17
10/21/2019 CSCI 561 (139 unread)
Also, how to measure 'further away' in 1c is not mentioned.
1a and 1c seems sufficient to ensure that each player eventually moves all pieces out of their home camps. 1b seems unnecessary...or am I missing something?
Also if 1a and 1c are enforced, then the rule for winning the game no longer needs to be modified, right? Daniel Link 12 days ago You're right about the wording of 1a.
"Further away" means in terms of increasing either the vertical distance without diminishing the horizontal one or vice-versa, or both. 1b exists because a player could still build a wall outside their camp, see above.
Daniel Link 12 days ago I was wrong about 1b and got rid of it, check the updates.
Resolved
Unresolved
Sourish Banerjee 12 days ago
Also, rule 1c invalidates several legitimate opening strategies, so it might not be the best idea to enforce it.
There are some posts online which propose a 50-move rule, wherein if a player does not clear his/her home area within the first 50 moves, he/she loses the game. This rule combined with 1a should ensure that each player eventually clears his/her home camp (or forfeit the game), while at the same time requiring minimal changes to the game rules and not invalidating legitimate strategies. Maybe this can be given some consideration?
Daniel Link 12 days ago We have to prevent exploits and therefore we can't really take into account what well-meaning players consider "legitimate". Dealing only with the camps is not enough because a player could still build a wall outside their camp, see above.
We can possibly consider slowing down the mandatory emptying out of the camp.
Sourish Banerjee 12 days ago Ah ok. The wall idea is not something that I considered...
Daniel Link 12 days ago I was actually wrong about the wall, please check the updates to the proposal.
Resolved
Unresolved
Sourish Banerjee 12 days ago
I think there is another way to spoil the game, i.e. entrapment of an isolated opponent piece (see attached image). As long as the trap is in place, the opponent can never move the trapped piece and can hence never win. This maneuver is arguably more difficult to execute but is still possible in certain situations. Are we allowing spoiling of this kind?
https://piazza.com/class/jzob0qt1owq15p?cid=334
5/17
10/21/2019 CSCI 561 (139 unread)
https://piazza.com/class/jzob0qt1owq15p?cid=334
6/17
10/21/2019 CSCI 561 (139 unread)
Resolved
Anonymous 12 days ago It is possible, however the controller of red chess do have oppotunity to avoid their chess being trapped. So I would accept that the player that create the trap wins the game, even if through shady moves
Daniel Link 12 days ago As of right now, I would say this is okay. All three cases are just responses to poor gameplay of the red player. Case 1 would bind 16 pieces of the green player and be really tedious to set up with a very small chance of winning while the red pieces can march unobstructed to the green base.
Unresolved
Sourish Banerjee 12 days ago
Do we still need the updated rule for winning the game? Because the players must eventually move all of the pieces out of their respective home camps, right?
Daniel Link 12 days ago The clarification is just so no one thinks they already won the game at the outset.
Sourish Banerjee 12 days ago I didn't mean the clarification. I think we can revert the winning condition back to being 'occupying all the opponent camp
squares' vs 'only occupying those that are not occupied by opponent pieces'.
Daniel Link 12 days ago It doesn't make a practical change and you've seen some of the reactions here when the slightest thing gets changed :). I'd rather keep everything the same where at all possible.
Sourish Banerjee 12 days ago Makes sense.
Resolved
Unresolved
Anonymous 12 days ago
so does this mean we cannot backtrack? like lets say one of our pieces is stuck in an L shaped wall, one original solution would be to backtrack to get out of the wall, but the new rule seems to prevent that.
Daniel Link 12 days ago You can only not go back into your camp.
Resolved
Unresolved
Anonymous 12 days ago I still feel the new rule 1.b) will cause problems as you would have to remove all pieces from the camp first. This will result in all pieces just outside the camp and will prevent lot of jumping strategies as all pieces would be cluttered in one area and also I have to waste moves in moving my innermost piece outside the camp whereas I can use those moves to advance towards enemy camp.
https://piazza.com/class/jzob0qt1owq15p?cid=334
7/17
10/21/2019 CSCI 561 (139 unread)
I think there should be some flexibility for this rule like moving a piece out of camp every 3 turns or so. This will ensure good play and flexibility and also prevent spoiling as well.
Daniel Link 12 days ago Then everyone would have to implement keeping track of moves, which is more error-prone than implementing what we're discussing.
Resolved
Unresolved
Anonymous 12 days ago
So according to this, we first have to move all our pieces out of the camp (and they can never go back in) and then continue with the rest of game?
Daniel Link 12 days ago Yes, that's right.
Resolved
Unresolved
Anonymous 12 days ago
@TAs can you please summarize the entire changes and send out a mail with updated HW2 question. Following up on Piazza is difficult as with every comment the problem statement is getting a different meaning. It would be helpful if you could re-circulate HW2.
Daniel Link 12 days ago We'll send out an announcement once this has settled, probably in the next 24 hours.
Resolved
Unresolved
Daniel Paul Pena 12 days ago
I have a solution that would avoid having to make changes in the movement rules:
Why not make a hard limit in the number of plays a game can have. For example, a game that goes above 350 plays for each player is likely being spoiled (that or player are just playing random), the game could enforce that limit and declare a winner based of which player is closer to the objective. This has 3 advantages:
- First, dont change the rules of the game from the agent perspective.
-Second, punish a spoiling strategy, the spoiler player would always lose because would be far from completing the game. - Third, don't limit the opening options of the players
Daniel Link 12 days ago Then we'd have an open and long discussion about what "closer to the objective" means. It makes more sense to have rule changes that are easy to understand.
Sourish Banerjee 12 days ago 'Closer to the objective' is something that is non-trivial to measure. Simple distance-based measurements will not work as jumps may allow pieces to move large distances depending on the state of the board.
Daniel Paul Pena 12 days ago Manhattan distance from all pieces to the oposite corner would be a enough. I'm talking of just applying this condition for games that extend for much more plays of the reasonable, so a spoiler player can't just block till kill the oponent clocks.
Even if you just move forward your pieces 1 by 1 without jumps you won't exceed 300 movement, so reaching 350 is obvious that somebody is wrong in the game, meaning that somebody is clearly spoiling the game. To spoil the game you have to keep a lot of your pieces blocking your own camp, so they will lose in that
https://piazza.com/class/jzob0qt1owq15p?cid=334
8/17
10/21/2019 CSCI 561 (139 unread)
Resolved
aspect.
This would be very easy to implement for the TA side, and it would be totally fair and will only apply to spoiled games.
Sourish, I don't think it makes sense to think about the jumps in these cases, because the closest to the obejctive function will be just to detemrine which of the players is spoiling when is clear than somebody is doing it.
However I understand the solution that the TA team is proposing, and proabbly will work and avoid most of spoiling cases. I just wanted to discuss another possibility.
Sourish Banerjee 11 days ago The non-spoiling player might run out of time before the 350 move limit is reached, so you can't just selectively apply this win condition only on reaching 350 moves...you must also do this in case of timeout.
And you can't enforce this on timeout, since then an agent's optimal strategy could be to intentionally timeout whenever it detects that it is ahead as per the Manhattan distance heuristic.
Also, as previously mentioned, Manhattan distance is not an accurate measure of being 'closer to the objective'.
Daniel Paul Pena 11 days ago Hi Sourish, I don't think a well design agent should have problems to play for 350 plays. The problem of spoilers is that the game goes virtually to infinite number of plays, maybe it will take several thousand plays to kill the oponent.
I agree that enforcing on timeout doesnt make sense.
Why Manhattan or another distance function wouldn't work? Just imaging the case of a spoiled game where one team is blocking the other team, spoiling the game, one of the players will have all his pieces next to the enemy camp while the other will be have at least a number of pieces creating a block in its camp avoiding the end of the camp, is clear that the spoiler player will always have a much higher distance to the oposite corner. Maybe I'm wrong but I can't think of a configuration when this wouldn't work.
Anyway, this is not going to be implemented so don't really matters i guess
Sourish Banerjee 10 days ago 'Well designed' is subjective, and the win condition must be designed taking into consideration all types of agents right? You can't just assume that agents will be 'well designed'. I'd argue that an agent that takes a little bit more time but makes good moves is better designed than any spoiling agent. But your win condition favours the spoiling agent over such agents. Also, what if two not 'well designed' agents are playing with neither agent spoiling, but the game crosses the 350 move limit. Is it fair then to assign one as the victor based on Manhattan distance?
Anyway, you're right...no point debating about this anymore I guess 🙂
Unresolved
Botong Lyu 12 days ago
The main goal of this rule change is prevent spoiling strategies. What spoiling strategy does is to force a draw and result in breaking that draw by having more time left. Therefore I feel that changing game rules do not directly address the general purpose of these strategies. Instead of making new rules for every new spoiling strategy that comes up, I think we should have a better method of ruling in case of a draw so that spoiling becomes harmful. Below is the proposed winning condition.
1. If any player's base is fully occupied with at least one enemy piece, the player loses immediately.
2. If any player's time run out, perform a check to see which player has more empty grid in their base. The player with higher empty grid count wins.
https://piazza.com/class/jzob0qt1owq15p?cid=334
9/17
10/21/2019 CSCI 561 (139 unread)
3. If any player's time run out, and they have the same number of empty grid in their base, the player with more time left wins.
This change does not affect the general game play and deal with all strategies trying to force a draw. Any wall building strategy or entrapment with result in more pieces used than empty slots gained. This way, we do not have to bend over backwards for every spoiling strategy that comes up and we do not touch the rule which, when changed, could result in other new spoiling strategies.
This also give a hint towards the eval function in the end game states without giving too much information about early and mid states.
Atharva Khare 12 days ago Problem with 2 is that an agent may empty the base asap and deliberately time out in next move, causing a forced win if opposing agent does not have the same strategy.
Botong Lyu 12 days ago That would be a problem.
Resolved
Unresolved
Anonymous 12 days ago Given the fact that you are changing such major rules more than half way into the assignment, it is very very unfair to people who already have a majority of their code done. If the rules are so important so as to be enforced now, they should have been thought about before giving out the assignment.
As far as spoiling is concerned, why don't the TAs write a separate script just to check whether spoiling has been done or not as a spoiling agent would always result in the game being a draw!
I propose you either don't change rules or extend the deadline for the assignment.
Nathaniel Potrepka 12 days ago Agreed. These rules feel arbitrarily contrived and ruin many good Halma strategies.
Daniel Link 12 days ago The rule changes are easily implemented. Having a generalized spoiling detector is not feasible.
Resolved
Unresolved
Anonymous 12 days ago
Firstly I feel that these rules that are changed in between have to be communicated officially in an email. We may loose out on some important stuff if posted locally on pizza.
Daniel Link 12 days ago They haven't been made yet, we're discussing them right now.
Resolved
Unresolved
Nathaniel Potrepka 12 days ago
1b is not a good rule because it ruins many good and legitimate Halma opening strategies! You may as well not call it Halma anymore.
https://piazza.com/class/jzob0qt1owq15p?cid=334
10/17
10/21/2019 CSCI 561 (139 unread)
Nathaniel Potrepka 12 days ago
Resolved
Daniel Link 12 days ago It affects both sides the same.
Nathaniel Potrepka 12 days ago Yes, it is a fair rule in that sense. My point is that the above is a good opening for blue in Halma; however, according to these new
anti-spoiling rules, it is invalid. This is now a different game.
Daniel Link 12 days ago You could argue that even with the existing anti-spoiling rules it's already a different game. We need to have a simple way to prevent spoiling that is easy to implement.
Unresolved
Nathaniel Potrepka 12 days ago
What should white do in this situation? According to 1b as it is currently written, white has no legal moves.
Edit: White can move. Read below.
B B B W W........... W W W W W........... W W W W ............ ................ ................ ................ ................ ................ ................
https://piazza.com/class/jzob0qt1owq15p?cid=334
11/17
10/21/2019 CSCI 561 (139 unread)
Nathaniel Potrepka 12 days ago In this situation, I don't think either player has any legal moves. EDIT: Black and white can move. Read below.
B B B W W........... W W W W W........... W W W W W........... ................ ................ ................ ................ ................ ................ ................ ................ ........... B B B B B ........... B B B B B ........... B B W W W ........... B B W W W . . . . . . . . . . . B B W. .
Resolved
Nathaniel Potrepka 12 days ago Never mind! I did not read carefully enough. "Only if the player does not have any pieces left in their camp or none of the two alternatives above are possible are they free to move pieces outside of their camp."
Unresolved
yujie sun 12 days ago
Shouldn't 1b.1 and 1b.2 be a or relation, that is, a player can either move a chess outside the camp or move a chess further from the center. Because in the current rule, you have to move outside chess first and you can not move chess in the corner, which is a bad strategy.
Daniel Link 12 days ago I assume that by ¡°chess¡± you mean ¡°piece¡±. In any event, you¡¯d need to be more specific than calling something ¡°bad strategy¡±. It¡¯s an unavoidable side effect of any rules changes that some strategies won¡¯t work anymore, sorry.
yujie sun 12 days ago I am just making some suggestions. I think making it an or makes more sense and it will be easier to implement.
................ ................ ........... B B B B B ........... B B B B B ........... B B W W W ........... B B W W W ...........B B W W.
https://piazza.com/class/jzob0qt1owq15p?cid=334
12/17
10/21/2019 CSCI 561 (139 unread)
Resolved
Unresolved
Anonymous 11 days ago
When are we going to know if/what new rules are going to be officially applied? I don't think it can be assumed that all students have read this thread and it is unclear whether this constitutes as an official change.
Bo Wang 11 days ago If you are asking whether it's officially announced, the answer is no. Sorry..we are waiting for a final check.
Resolved
Unresolved
Anonymous 11 days ago
I just want to check before starting: Is this the final version of the rules?
Bo Wang 11 days ago If you are asking whether it's officially announced, the answer is no. Sorry..we are waiting for a final check.
Resolved
Unresolved
Anonymous 10 days ago
Can you please provide example based on updated rules
Daniel Link 7 days ago We think the rules should be clear enough. If you still have specific questions, please post them here.
Resolved
Unresolved
Anonymous 8 days ago
what if this happened?
That the one black piece in the red circle can't move and we can't move any black pieces.
Because since there is still one piece left in the camp, we can't move the pieces which are not in this camp. But we can neither move this out of camp nor move it further away from the corner [0,0].
https://piazza.com/class/jzob0qt1owq15p?cid=334
13/17
10/21/2019 CSCI 561 (139 unread)
Anonymous 8 days ago Quoted from the addendum,
https://piazza.com/class/jzob0qt1owq15p?cid=334
14/17
10/21/2019 CSCI 561 (139 unread)
Resolved
Only if the player does not have any pieces left in their camp or none of the two alternatives above are possible are they free tomove pieces outside of their camp.
which means that you can move pawns outside of their camp in your
case, since there is no way to move the only black pawn inside the camp to somewhere out of camp, our further away from the conner.
Anonymous 8 days ago thank you! I did not read carefully enough..
Unresolved
Anonymous 3 days ago
Hi. I just want some clarification regarding HW2 addendum Rule 1b. first bullet point.
Move a piece out of their camp (i.e. at the end of the whole move the piece ends up outside of their camp).
In the case I end up with a piece within our camp. An example (15,15). Does this mean that with just one move I can move directly this piece from my camp (yellow zone) to the blue zone. Please see attachment.
S(n) state
S(n+1) state
https://piazza.com/class/jzob0qt1owq15p?cid=334
15/17
10/21/2019 CSCI 561 (139 unread)
Daniel Link 3 days ago Which move would that be?
Anonymous 3 days ago Note: I didn't draw my other pieces, but I just want to clarify that by that time the rest of my pieces are outside the camp and 15,15 is the
only piece left in my camp.
Daniel Link 3 days ago I get that, but can you give me the notation of that move?
Anonymous 3 days ago This is the confusing part for me: at the end of the whole move the piece ends up outside of their camp. which I'm translating as the only possible play for me is to move that piece to the bounds of my camp.
Daniel Link 3 days ago Notation please.
Daniel Link 3 days ago I guess I'm not going to get a reply. In any event, you're misinterpreting the rules here. All priorities only apply to actually possible moves.
Those can only either be moves to an adjacent position or sequences of jumps over one more pieces. The move you're showing here is never possible.
Anonymous 3 days ago Sorry I didn't get what kind of notation are you looking for. But your last reply answered my question. Basically the only possible moves for a 15,15 piece if it is the only remaining piece in my camp would be (14,15) (14,14) and (15,14)
Daniel Link 3 days ago Right. Anonymous 3 days ago Thanks
https://piazza.com/class/jzob0qt1owq15p?cid=334
16/17
10/21/2019 CSCI 561 (139 unread)
Resolved
Unresolved
Anonymous 8 hours ago Hi,
Wanted some clarification regarding winning the game.
Addendum states:
"2. Winning the game (no real change here, just a clarification)
A player wins if they are the first to fill out all the space in the opposite camp that¡¯s not occupied by the opposite player¡¯s pieces using at least one of their pieces."
Also i read somewhere that if our time runs out, even then we loose.
Does that mean even if we have 18players in opposite camp, and opposition has none - and our time runs out - then we lose? Isnt it unfair!
Bo Wang 8 hours ago @275
https://piazza.com/class/jzob0qt1owq15p?cid=334
17/17